Search code examples
c#.netdatabasescalabilityreliability

Search for information on building large enterprise systems

How do you organize DB layer, business logic and cross-platform API of your information management system, if uploading and processing 500000 data records in one session is a normal operation (C# .NET 3.5 + MS SQL 2005)?

I’m specifically interested in production-proven paging patterns that behave well with the concurrency, scalability and reliability.

Does anybody have any ideas, in what direction to dig?

  • Open Source Projects (don’t care about the language or platform, as long as it is not Ook)
  • books
  • articles
  • Google keywords
  • forums or newsgroups

Any help would greatly appreciated!

Update:

  • simple paging (i.e.: rownumber in SQL 2005) does not work, since there are a lot of concurrent changes to the database. Item, that is deleted or inserted between the page requests, automatically makes current page index invalid.
Solution

  • Done the implementation. I have been informed recently that one of the uploads was about 2148849 records. Tiers did successfully deal with the a couple of broken connections and dozens of deadlocks at the DB level during this upload.

    In case somebody else needs some info: