Reading in the Industry blogs, official samples and documentation provided with MS SQL Server 2005+, Visual Studio 2008 and Linq to Sql and the Entity Framework, you will get the impression that we have a tremendous ability to efficiently decipher how to interact with a Database. Additionally, Industry leaders have led us down a path of Code Generators, Designers and the separation of UI Design from functional Object code; read "Astoria" and/or "MVC" here...
Didn't CASE do this sort of thing for us years ago? Didn't it also fail? There is huge amount of traffic about the topic of "Falling down when you hit a threshold" and insistence that all things should be built ultra-scalable. OK, that's great if you love to write plumbing. I for one, don't. What if that could be done for you, with the most efficient options always at your disposal; of course with the opportunity to tweak and extend to meet a specific need.
While there are many great arguments to be had over what is exactly ultra-efficient and those who think they have a better method must be able to override and extend the selected generation of code that is offered by not only being able to write that code, but also to inject it into the Generated Code (or at the least allowed to edit the templates) produced by Visual Studio.
If you don't already know why this is so important, we need to first look at the massive shift in Business Software Development and where we are going to be running most business applications. We used to run all our apps on our own hardware, inside our own network and everything was under our tight control. While this model still applies to much corporate development, I believe it will become less and less used as a cost effective alternative to a HOSTED solution. Pricing of hosted solutions have plummeted in recent years, in fact I am almost willing to bet that the cost of electricity alone using hardware just a few years old would cover the cost of a hosted solution. And what happens when I DO need to scale, How do I add 20 servers to the Web Farm overnight? Staffing requirements and maintenance of hardware start to become totally offset when we look at this model. I CANNOT BUILD A SERVER CHEAPER THAN I CAN HOST ONE! That is the real bottom line.
I would wager that MOST applications would run just fine on a hosted solution with one machine doing all the work, at least in the beginning, or at least with a hoster that gives you Web on one machine and your DB on another. Are we talking about every solution here? No, but I think we are talking about a gigantic sector of the market that is currently all but ignored by the provided tools we have at our disposal.
Don't get me wrong here, we have GREAT tools available and more coming, do they target what is really needed?
First lets look at the three new primary tools given to us by Microsoft. WCF, Linq to Sql and the Entity Framework. WCF provides a great pipeline to pass data between our UI, Business Objects and Database, but it just plumbing. Linq to Sql and EF provide a great way to interact with a Database IF and ONLY If you have a proper database in place first.
What does this mean to most data-driven development projects? It means they either need a DBA, or they need to hire one for the portion of the application that defines the database. This is one reason Migrations for Ruby on Rails has gained so much popularity, it hides this chore. Most of us don't care about the DB, it's just a place that holds data. Some of you can go on and on all day about the subtleties of databases but that is completely outside the scope of what I am talking about here. Not only will Migrations create a DB and Schema around what your App needs, it will also keep it up to date with changes made by any developer that can be easily stored in Version Control software like SVN or TFS.
This is what the Designers for Linq to Sql and/or Entity Framework need to do for us. I really don't mind changing my Linq to SQL Apps over to Entity Framework if it will allow us to focus on one great technology. It's great that they can operate against a currently created database, but they should also create one (IN AN OPTIMIZED WAY) when one does not exist. It should create Stored Procedures if I tell it to... Even if we are migrating a legacy application, I would still like to see what a tool spits out for me. I just might use the new techniques. There are tons of ways to move data from one schema to another, again, outside the scope of this article. So back to the scope... EF Does a great job of creating an interaction between Database and Business Objects, but it doesn't help us rapidly create a NEW model without an existing database and I think this is important in two ways, first and foremost, I don't want to write any plumbing code to do this. The tools are smart enough already to do almost all the work for us, it can go just a little bit further and create the expected interaction that solves most of the interaction of the database work for us. I can create a Model with the EF Designer by adding Properties and Associations, but it won't go and create a database around that model.
I should not have to know about how the interaction happens, just that it does and that it does so in a fairly efficient manner. If I lose a certain percentage of performance due to layers, then so be it if I can create my application 10 times faster. If the tool developers choose to implement change tracking in a DataContext or Object Context that is great, do so in a way that makes sense for the CODE developers to use, not require me to go hire Database professionals to argue about the proper plumbing or storage techniques. Linq ALREADY knows how to create efficient SQL, if it did so in code generation vs. dynamically at runtime who cares?
We can host a solution out in the cloud so should we need to be worried about the optimization of how our application persists its data? Certainly there is is need to cover legacy applications and all the DBAs running around out there, that is well covered right now. What we seem to be missing is the ability for competent application developers to create Rich Internet Applications (read Silverlight and Ajax) and have the data persisted without a bunch of plumbing and tweaking.
What is NOT covered is the ability to rapidly envision, design, develop and deploy solutions destined to live on a Hosted Solution -- even if it is "hosted" internally. This solution will most likely have a distinct separation between UI, Business Objects and Database, but do I really need to see those distinctions in my code? Not really. Do I really need to spend the time to wire up all the Databinding when the MetaData already knows what I need to do? No Way, a tool can create that for us. It can already look into our metadata and see what we need to do to hook up a DataGridView to a table.
I would venture to say that the tools are already here, they just don't work the way we need them to so we can build these applications quickly. I spend MOST of my time Creating a Database, then working on plumbing and interactions between the Database and the UI, I should not have to do this any more. Microsoft did a fabulous job of giving us a database that can scale with us as our application grows. We have SQL CE, SQL Server Express and SQL Server Enterprise to grow with us, we can use SQL CE for Mobile Apps and offline stuff, wouldn't it be wonderful if I did not have to architect that portion of the solution any more? I am talking about the people who do not have the luxury of a full time DBA.
Of course, the solution is there, staring us right in the face, the answer is the Designer for Entity Framework. If this Designer could be used to build out the Database, and track the Schema changes, then it would take down part of the battle, just a little more work and it could completely remove the NEED (we should always have the option to extend and modify though) for us to write anything between the Business Objects and the Database.
We should not need to write Serializers for Silverlight, WCF or anything else, the tools already KNOW how to do that, they just don't. The MetaData is there, let's use it. Rob Conery's Subsonic and ActiveRecord shows there is interest in such a tool, but I say this tool needs to come from Microsoft and it needs to be built into the current generation of tools we are using or are being pushed to use (aka Linq). With Rob and Phil working AT Microsoft why can't they contribute some of that knowledge over to the ADO team and get this into the EF Tools we need. Pablo, Mike, Dinesh and the rest of the ADO Team are doing a great job so far, but they are only touching on what we really need. The newest version of EF finally gets a way to do disconnected graphs which is a step towards the goal thanks to Danny Simmons.
These tools need to work seamlessly with Linq and allow me as a developer to write applications with little or no concern for how my data is persisting. I want to create Controls that I can offer to my UI which work seamlessly with Ajax or Silverlight and not require me to write a ton of plumbing just to move things back and forth from one layer to another.
This is the challenge for the new world of development. We have already moved to a hosted world and our tools need to move there with us. But what do I know, I am a VB developer first and foremost... This is what I know, my customers are asking me to create these applications and to create them really fast. I can choose to use the tools I know best or I can branch out and find something new. The problem here is that I have already invested years of time to learn the tools I am using and they are simply not doing what I need them to do to make my job truly easier.
My Invoice Object doesn't really need to know what DataContext my Customer Object lives in and I as a developer should not have to be concerned about that in the slightest. I should be able to freely take a Customer I loaded from Invoice "A" and assign it to be used for Invoice "B" without worrying what context it came from, Invoice "B" just needs to know its a valid Customer or that it is a new one I just created without me forcing something into the plumbing to test it. It should know how to roll itself back when I cancel changes, etc.
With Generics and XML Literals, VB is a Top Notch Code Generator. Kathleen has proven that here and Karl talks about Metadata and what we can do with it over here... We don't need Yet Another Code Generator. We need something that works with the existing modelers and designers and just makes our life as developers easier. It needs to be INSIDE Visual Studio and we need to be able to work with the templates. I should not need LLBLGen or MyGeneration or CodeSmith or SubSonic to do something Visual Studio pretty much already knows HOW to do, it just doesn't. I don't need another designer that works with an existing database, I need the CURRENT Designers to Make the Database AND the plumbing.
I personally think this should be the direction for the next revision of Visual Basic. VB IS the glue that puts the framework together, sure you could also do it in C# (if they added XML Literals...) but VB is already geared for it so why not enhance it's productivity even further by giving us a way to be far more productive than we are today. This does not require any great shift in the tools we already have, it just requires Microsoft to use them smarter and provide us the keys we need to open the door.