I already know there are numerous threads outside discussing the use of section elements - I already read a lot of them and so far I came to the following conclusion:
A <section> element should be used to wrap content which could be stored as an individual database record. (according to HTML5doctor) and that the heading should be inside the section element (although in a database I wouldn't store the heading in the same column as content...))
So far so good, I reorganized my website's structure and had it analyzed by http://gsnedders.html5.org/outliner/ - but surprisingly it kind of ignored my h1, h2, ... headings order (somewhere I read that special elements reset the headings or so..? :S) which kinda drives me mad.
By now I've got a structure like (skipping some div's for styling purpose):
<header>
<section>
h1 // title of my page --> h1
</section>
<section>
<nav>...</nav>
</section>
</header>
<div role="main"> // don't use section for your website's main content
h2 // main content
<section>
h3 // main content's sub heading
</section>
<section>
h4 // sub heading of main content's sub heading
h5
h5
</section>
<section>
h3 // another main content's sub heading
</section>
</div>
<aside>
<section>
h3 // about me
</section>
</aside>
instead of the expected outline:
title of my page
navigation
main content
main content's sub-headings, etc.
about me
I got:
main content
title of my page
navigation
main content's sub entries
about me
of course I could wrap my headings in some section elements to reset this counter, but I would never store the entire main content as a single database entry - so this would be an unsemantic usage of a semantic element, which makes the whole work somewhat redundant.
Did I miss the point of the section element or the nested headings? Any proposals how to handle this behavior (which sounds great in theory, but in practice it seems to be behave in a strange way...)
EDIT: I decided to stick to the "use article on elements that could be a RSS item" rule, as it makes much more sense to me and went on to use a section element to keep the main content apart from the navigation etc. (as answered below)
The section element
I'm not sure if I agree with your section definition. From the HTML5 specification it says
The section element represents a generic document or application section.
It goes on to say
Authors are encouraged to use the article element instead of the section element when it would make sense to syndicate the contents of the element.
So I think an article should be used in place of section if you are going to store something as an "individual database entry". See also this good list of DOs and DON'Ts for the section element.
Document outlining
In terms of your document outline, it is difficult to tell what is going on as you have not included any closing tags. Can you post a complete, valid HTML example?
There is an excellent overview of outlining in HTML5 documents on the Mozilla Developers Network which I recommend you read. Taking your example and adding the closing tags where I think they should be, and based on this article, I would propose the HTML
<doctype html>
<header>
<h1>title of my page</h1>
</header>
<nav>My Nav</nav>
<article id="main-content">
<h2>main content</h2>
<section>
<h3>main content's sub heading</h3>
<section>
<h4>sub heading of main content's sub heading</h4>
<section>
<h5>a sub, sub heading of main content's sub heading</h5>
</section>
</section>
</section>
<section>
<h3>another main content's sub heading</h3>
</section>
</article>
<aside>
<h3>about me</h3>
</aside>
This produces the outline
Is this the outline you expected?
Section headings
Note you don't have to use h1, h2, h3 etc., the HTML5 outline will work out what is a heading, a sub-headingetc based on the depth of nesting of your article or section. You could, if you wnated, use h1 everywhere (although don't) and the outliner would still produce the above outline.