Posted on 1/29/2009 10:29:19 PM by Justin Etheredge
I don't know how most people who attended tonight felt, but tonight's meeting was exactly as I had envisioned it in the beginning. We had a large number of really passionate developers who came together to have a discussion about a topic that is important to them. So basically, the meeting was awesome and I think most people really enjoyed it.
Going forward we are going to rename the group to the "Richmond Software Craftsmanship Group". We feel like the name lets people know exactly what we are about. We have a google group setup and ready to go, and the initial members who attended tonight will get their invites soon. You can check out the Google group here.
Posted on 1/27/2009 6:55:59 PM by Justin Etheredge
I don't know about you, but I used to love those little painting by numbers kits when I was a kid. For those of you who have never used them, they had pictures that were broken down into different shapes, each with a number in it. Those numbers in turn corresponded to a particular color, and all you had to do was match a color with a number and fill in that shape with the correct color. Here is a pretty extreme example of what I am talking about:
When you were done, you had a wonderful picture, without all of that nagging art talent! In fact, the creator of the picture was the one with all of the talent, all you had to do was just keep your paint (or crayon, or maker) within the lines and in no time flat you looked like you were actually capable of something! The problem was that the process of filling shapes didn't correlate at all to the process of actually painting something. It is a bit like your friend who plays the guitar on Rock-Band and decides that he should now take up guitar. You know him, we all have that friend.
With paint by numbers though you didn't need to understand color theory, shadowing, perspective, etc... All of the things that you must understand, whether explicitly or implicitly, in order to be a good artist. And to top that all off, no matter how long you did this you never got any better at drawing a real picture. Since the skills didn't translate, you could paint by numbers for the rest of your life and still produce paintings that looked like nothing other than Jackson Pollock knock-offs.
All of this may come as absolutely no surprise to you at all, but as developers we do this all of the time. Consciously we know that programming does not equate to application architecture, but how many times are designs handed down from ivory tower architects to their underlings in order for them to "Program by Numbers"? Being able to fill in a method, or put a method on a class will never allow that developer to understand the intricacies of object oriented design or the complexities of system design. Developers can write code day in and day out for years, and still end up being absolutely terrible at writing software because they have never been given the training they need in order to learn the skills of design.
But we, as developers, should know better. How often do we complain about sub-standard code? How often do we complain about code that we wrote two months ago? How often to we complain about horribly architected systems? If we want to keep acting like what we do is a craft, then we need to start treating it like one. And that means training and sharing the knowledge of your craft with others...either a direct or indirect apprenticeship. More experienced developers can't just sit around and expect college grads to come into the job with all of the knowledge they need in order to write excellent software. Just think about how much knowledge you have garnered since your earliest days as a green developer! If we want these new developers (and many old ones) to get better then we need to get rid of the numbers and start letting them think (and screw up) for themselves.
Posted on 1/23/2009 12:31:09 AM by Justin Etheredge
(This post is meant to be a tad humorous, so take a deep breath before you read it)
I'm writing this post because I am getting tired and weary of people pointing to "drag and drop tools", "visual designers", and "wizards" as reasons to use a particular tool. Only in Microsoft-land are developers so enamored with "wizards" and "drag and drop" and I think this is because Microsoft is the proverbial "crack dealer" when it comes to providing these tools.
The problem that I have with these tools is that they are designed to get you up and running quickly, but often to the detriment of your later development. How many of the tools that you use which have wizards or visual designers let you go back and edit what is produced? And if they do let you edit what is produced, how many of these tools will then let you go back and use the wizard or visual designer after you have made modifications? I would be willing to bet very few.
And lets think about why this is.... how come a visual designer can map out a complex process or schema, but then is unable to let you edit it? The simple answer is that our textual languages are far (far far far far) more expressive than any visual designer could ever be. Think about that for a second. Let's use the Entity Framework as an example, (I hate to keep picking on it, but I have been dealing with it a lot lately) the graphical designer in the Entity Framework is one of these tools that is optimized for getting you up and running quickly, but this is done at the detriment of later flexibility.
With the Entity Framework, assuming that I already have my database schema complete, I can point the graphical designer at the database, generate my entities, and I am off. At this point I am ready to start creating and saving entities. This all seems great until I realize that I am severely limited by the generated code the EF spits out. (In VS 2010 they are planning to let you use T4 to customize the generated classes, but I'm not sure that this really solves the issue, it just ties me to different code) If I do decide to edit the generated code, then I lose the designer. So what if we do lose the designer? Is that all bad? Well, it wouldn't be if the tools weren't designed around having the mapping files and code generated.
I may take some heat for that last statement, but if you look at the mapping files, build tools, etc... You will start to see that the EF was never really designed to be used without the tools they have created. Think about it, because of the fact that they assumed that tools would be used, they put in place constraints that never would have been accepted otherwise. They descend from a standard base class (which is technically optional), they require that strings be passed for property change tracking, there is no automated change tracking, relationships require that you pass in foreign key names and entity set names, they require a custom key to be on your entity, you can't use factory methods, etc... If they didn't expect for the EF to be used without their tools, then they sure created a headache for developers that don't want to use them. They allow for the EF to be mostly used without a lot of the generated code, but they certainly didn't optimize for this scenario.
In effect what happened is that the Entity Framework is optimized for the start of your project. They assume that you have your database schema all mapped out, then you point the tools and bam! Sorry, I got a little Emeril in there. So you've stood this project up really fast, but I would say that the vast majority of the complexity in most projects comes out of the complex business logic and processes that are introduced into it, not the basic layout of the domain entities. As time goes on and you need more and more logic and flexibility in your domain layer, the last thing you want to do is be held back by your persistence technology.
The solution that some people have pointed out is to simply use EF or Linq to Sql generated objects as DTO's and then map the data back onto your business objects. This actually seems to work okay in many cases although it seems to me that we are doing a lot of work for the ORM that it should be doing itself. If you are looking into this solution, then you may want to check out the December 2008 issue of MSDN magazine where there is an article titled "Flexible Data Access With Linq To Sql and the Entity Framework".
To illustrate the importance of not optimizing for the beginning of the application construction process we are going to start off with a completely ridiculous and overly simplified graphic:
This is what I call the software product construction pipeline. And like I said, it is completely ridiculous, and totally not drawn to scale. What would it look like if drawn to scale? Well, probably something like this:
I would have made the "beginning construction" part even smaller, but I figured I'd leave a little bit in there for argument. Now keep in mind that I am just describing the actual construction part of the application. You may be wondering what happened to the "Ending Construction" piece from the first graphic. Well, I forgot to tell you, but that part doesn't exist. No project is ever really finished.
So now that we are agreed (well, you better agree) that the beginning phase of our construction is much smaller than the total of construction, lets go ahead and describe what "beginning construction" is. Doesn't this sound like the way that most projects are planned out? Beginning construction is about getting the environment up, getting basic layout planned, and getting to the point where you can really start digging in and developing your application. "Construct" is all about building the thing which you set out to build.
So what part of the process do you want to optimize for?
This guy...
Or this guy?
I trust that you'll make the correct decision.
Posted on 1/22/2009 5:21:56 PM by Justin Etheredge
After a long winter siesta, the Meet and Code is back in full effect next Thursday! We are getting back to our roots by having an open discussion around testing legacy code. Al Tenhundfeld will be joining us to facilitate the conversation.
The meeting will be at 6:00 at Snag-A-Job's Headquarters in Innsbrook (the real meeting will start at 6:30):
4880 Cox Road, Suite 200 Glen Allen, VA 23060
Here is a little tidbit of info about Al:
"Al Tenhundfeld is a Senior Consultant with Dominion Digital, a Virginia-based technology and process consulting firm. He has worked in .NET since the 1.0 beta. In that time, he has served as DBA, data architect, and solutions architect for enterprise software solutions. He is active as an organizer and speaker at local .NET user groups and CodeCamps. His current focus is helping teams embrace mature software development strategies including DDD, TDD, and Agile methodologies."
Please go here to register so we know how much food to get.
Posted on 1/14/2009 8:03:00 PM by Justin Etheredge
Despite the title of this post, there will be no rapping or dancing of any sort going on here. Only talk of Windows Azure, which is quite possibly the most confusing product rollout that Microsoft has attempted since Vista. With Vista it was a plethora of versions... do I need Home, Home Premium, Business, Ultimate, Super Deluxe Extra Gooey edition???? I just don't know! Well, with Windows Azure the problem seems to be the opposite. It seems that they are trying to cram a ton of stuff under a single umbrella. In a way reminiscent of the direction that they originally went with .NET, and then later backed up on quite a bit. Did you know that Windows Server 2003 was going to originally be named Windows .NET Server?
Anyways, I have spent some time over the last few days really digging into Windows Azure and trying to figure out exactly what all of the stuff is in that massive slide that kept showing up at PDC. You can see what I am referring to about half way down this page. I don't want to reprint it here since I may incur the wrath of my mighty overlords.
First I am going to clear up what I think was the single biggest mistake they made when they created that image, and maybe I am an idiot for thinking this, but I had assumed that they were trying to show the five pieces of Windows Azure at the bottom. But they aren't really. What they are showing is five services that can be used in conjunction with Windows Azure. Windows Azure is really big because it is much more badass than the rest. :-) Of the five pieces that are sitting above Azure, we are going to completely ignore "Microsoft Sharepoint Services" and "Microsoft Dynamics CRM Services" because I don't really have anything to say about them. The three parts that we will look at are "Live Services", ".NET Services", and "SQL Services", but these aren't part of Windows Azure. They are part of the "Azure Services Platform", and that is what confused me. I can't imagine why that would be confusing!
Windows Azure is a separate service that is closely related with many of the other services list, but even Windows Azure is broken down into a few parts! Clearly Microsoft has invested ridiculous amounts of money, people, and whatever else Microsoft has (gold bullion?) into this.
Part 1: Windows Azure
Windows Azure is a hosted platform for writing applications that can scale easily. Inside of Windows Azure there are two very distinct different services that are running. The first is "Hosted Services" and the second is the "Storage Services". I say that these are distinct because they are almost entirely unrelated. Sure you interact with them inside of the same control panel, but their interactions and versioning are handled in very different ways.
Hosted Services:
You have a concept of a "Web Role" and a "Worker Role". You can think of a "Web Role" as an ASP.NET application, and a "Worker Role" as a Windows service. All that these services do is fire up .NET code in Microsoft's datacenters and allow for you to spin up an arbitrary number of each role in order to do your bidding. Microsoft seamlessly handles the rolling out, load balancing, etc... But this part of Windows Azure is really all about code, there is no way for you to store any kind of state. This is how they let you scale, because if you need more instances, just fire them up! If you have no state, then your roles shouldn't care where they are running. The problem is that without state, your application is probably pretty useless. This is where the "Storage Services" come in handy.
Storage Services:
This is another confusing part of the whole Azure stuff. We have a storage service inside of Azure, but then we also have "SQL Services" which isn't part of Azure, but it is a part of the "Azure Services Platform".... whaaaaa??? So yeah, there are two storage solutions. One is part of Azure, and that is Azure storage. The second one has little to do with Azure, and it is called "SQL Services". Azure storage is a very simple storage solution. You have three storage options "Queues", "Tables", and "Blobs". Queues are just that, a Queue. If you don't know what that is then you probably ended up at this page by accident. You can plop items on a queue and then pick items off. Tables are a very simple row storage mechanism. You can do some simple queries over the data, but you are very limited. The tables aren't relational so you can't have a foreign key to another table. There is a Linq provider, but it only supports "where", "select", and "take". Notice that there is no "skip", so you really can just retrieve items and you can't do any storage side paging. The tables also only support a small subset of data types. One of them is not "decimal". Yeah, no decimals. Ouch. Blobs are for storing things like images, files, etc... They can even be made public so that you can access them through urls just like you would any other web resource.
As you can see, it is the combination of the hosted services and storage services that allow you to actually do something with Windows Azure. As I said earlier, the blobs are useful for storing website assets, and the tables are useful for storing whatever data you need to drive your websites/service/etc... You may be wondering about the queues though, what kind of use do they have? Well, remember that we had both "Worker" and "Web" roles. The queues are the perfect place for a site or service to place work on the queue for a backend process such as a Worker role to deal with. Then the worker role can put another message on a queue, or update a piece of data in a table when it is done processing the work item. It allows for very distributable asynch processing of work. The kind of thing that you would normally use something like MSMQ for.
These two pieces are what actually comprise Windows Azure. There is one additional piece, and that is the web portal, but that is simply where you perform the administration and deployment of your applications. I'll give you a few quick screenshots so that you can see what I am talking about:
Here is the main project page that shows the different projects I have created:
Here you see that my "Hosted Service" project (the top one) and my "Storage Service" project are two entirely different things inside of the admin panel. If I go inside of the hosted project you can see that we have a "Staging" environment and a "Production" environment. We can initially deploy our application into the Staging Environment where it is allocated and started. Then we have a special url that can be used to hit this staging deployment to test it. Then when we are ready to roll it into production, you just click the blue button you see in the screenshot below and the load-balancer is points the production url to the staging deployment, and voila, you are deployed!
I'm not going to go into any further detail about this (maybe in a future post), but I just wanted to give you an idea of how you got your applications into the "cloud".
Disclaimer
For the rest of the services I am only going to give you a cursory explanation of what they are. Mainly because I have not worked with them to any extent, and secondly because right now I am mostly interested in the hosted Azure platform.
Part 2: Live Services
Live Services are another gargantuan offering that has even swallowed up a few services that already existed before this branding was announced. To sum it up in Microsoft's own words:
Live Services are a set of building blocks for handling user data and application resources.
Sounds pretty simple, until you see this guy. Holy crap. So this poster is actually about the "Live Framework" which is built on top of "Live Services" and provides a uniform way to access all of the "Live Services". Okay, so what are all of these services that you speak of? Well, I'm glad you asked. A few of them are "Live Alerts", "Live Contacts", "FeedSync" (Which Live Mesh is built on top of), "Messenger", "Virtual Earth", etc... there are actually quite a few more of them. Some of these are services that users can interact with directly, but virtually all of them are going to be interacted with through the "Live Framework".
Part 3: .NET Services
.NET Services are formerly known as BizTalk services, or at least part of it was. BizTalk services appear to have disappeared in a way reminiscent of Soviet era style cleansing. Can you say "persona non grata"? I remember hearing about BizTalk services at Mix 08 and the presenter actually said that he didn't know why they were called "BizTalk" services since they had nothing to do with BizTalk. Well, I guess Microsoft listened, and they have now been renamed. .NET Services provide three different pieces of functionality, and are the completely developer focused side of their service offering. The three pieces are "Access Control Service", ".NET Service Bus", and "Workflow Service".
The "Access Control Service" allows application developers to move authentication and authorization out of their application and into a hosted service. This allows users to authenticate into an application from anywhere using a set of well known services. This service also provides the authorization mechanism needed to interact with the rest of the .NET services.
The "Service Bus" is surprisingly not really all that much of a service bus. :-) If you were expecting something like MassTransit or NServiceBus then you will probably be disappointed. Think of "Service Bus" as more of a hosted replacement for MSMQ. It has a very simple pub/sub model that allows you to pass messages across the internet. I'm not saying that it is bad (in fact I have never used it), it is just made to be simple and to only provide basic messaging support. If you need an async hosted messaging system though, and don't mind writing some of the more complex features yourself then this may be an excellent option.
The last piece is "Workflow Service" which is a service that allows you to host workflows written in with Windows Workflow Foundation. It can be connected with remotely through http or it can use messages off of the .NET Service bus. Other than that I don't really know much about it, so I'll leave it to you to explore. It sounds like it might be neat to perform more advanced .NET Service bus interactions and routing using the Workflow Service. You could have the Workflow Service pick up messages at endpoints, process them, and then deliver them to other endpoints that are delivered to recipients. The idea though of having a hosted platform for distributed - long running processes is pretty enticing though.
Part 4: SQL Services
SQL Services are probably the only self-explanatory part of the entire Azure Services offering. It is SQL Server in the cloud! Mmmmmm.... cloud. SQL Services are an interesting beast. Just like the Azure data storge, they only support a small subset of types. SQL Services looks to support decimal, so it is all good in my book. The interesting part about SQL Services is how it is setup. It supports joins, so you'd think that it was relational, but it is not. In fact, it doesn't even have schemas! At the top most level there is a concept of an "Authority". In an authority there are "Containers". Inside of these containers there are "Entities". From this you may wrongly expect that Authority = Database, Container = Table, Entity = Row, but in reality the mapping is quite different. Authority is more or less like a SQL Server Instance and then a Container is like a Database. But an Entity is a table and a row? Yep, kinda.
The "Containers" could be thought of as a table, but since there are no schemas we can just put whatever kind of entity we want into them. So containers could hold both "Order" entities and "LineItem" entities. But why would we want to do this? Well, because remember earlier how I said that we could do joins, well they are only within a container. You may be screaming right now about this, but remember that SQL Services are meant to be distributed and scalable, if you have data in two containers, which may reside in two different services or data centers then how do you know how expensive joins will be? Microsoft hints in their docs that cross container queries may be possible in the future, but I'm guessing that they will have to do some kind of data syncing in order to enable this feature.
In the end SQL Services provide you with a bit more flexible data storage engine than what Azure provides out of the box. In the future Microsoft has plans to make SQL Services even more rich and bring it closer and closer to what we expect out of a full fledge relational database. I'm pretty excited to see where Microsoft takes SQL Services and it is certainly something that I will look more into in the future.
Wrap-Up
The Windows Azure services platform can be quite daunting to approach, but once you figure out that you can just tackle it one piece at a time it becomes much easier to manage. I for one have started to dig into Azure Data Storage and Hosting and will hopefully have a few more posts on this in the near future. Microsoft has clearly invested a huge amount of resources into this so I would imagine that you are going to be hearing quite a bit about it in the future. Also, if anyone out there is interested in potentially leveraging the Azure platform and doesn't know where to start, contact me because I am always interested in consulting work!