Posted on 6/28/2008 5:52:08 AM by Justin Etheredge
It seems as if everyone in .net land has chimed in on the whole Entity Framework "Vote Of No Confidence" petition recently, and up until now I have mostly stayed on the sidelines. As with any seemingly risky move, I was quite torn on the issue. And to a certain extent, I still am.
Let me first start off by saying that I think the ALT.NET community is a wonderful thing, and its efforts to enlighten Microsoft programmers to worlds and tools that aren't found on the Microsoft.com domain is quite laudable. I think that all of you are quite passionate about your craft and you truly seek only to enlighten others with the knowledge that you have gained from your own experiences. I consider myself to be a part of this community, because I honestly do feel like there is way too much of the "no one ever got fired for buying IBM" mentality in the Microsoft space. We, as a community, need expert guidance from individuals who are out there living these technologies and practices on a daily basis. A lot of us (myself included) don't have the luxury of working at a company that has total buy-in on TDD, BDD, DDD, Scrum, Lean, etc... Not all of us live in Florence Austin. Some of us have to settle for working at companies that have one foot in the pond, and we have to fight day to day to try and reach out in order to create an environment where other people will start to adopt some of our ideas. In this way, we can progress the quality of software in our own little corner of the world...and we desperately need guidance.
The problem is that I think that the message (and goal) is starting to get lost, or at least fuzzy. I look all over the blogosphere and read all the posts about the EF petition and it really makes me sad. I feel like I am in a microcosm of the U.S. presidential elections, and people have staked their claims on either side. No one is truly listening to each other, everyone is saying either "You are doing it wrong!" or "Why do you have to tell me I am developing wrong?". Some posts, like Steven Harman's, is calling out for everyone to go and sign the petition, while others, like Josh Holmes', are calling this the nuclear option.
You see comments in some of these posts like "In the midst of the discussion many valid criticisms have been drown out or lost in the noise of the cheerleading, trolling, and marketecture generated by community Gloryhounds, Redmond, and any number of other super-pro-Microsoft groups/identities." Does this jibe with the alt.net mission statement "We are a self-organizing, ad-hoc community of developers bound by a desire to improve ourselves, challenge assumptions, and help each other pursue excellence in the practice of software development"? Is calling people "Gloryhounds" and "cheerleaders" helping anyone to pursue excellence in the practice of software development? And yes, there is a lot of "Gloryhounding" going on. It will always go on as long as there is money to be made, but does it help anyone to start throwing around terms like that?
Now, I don't want to call Steven out on any of this, I know that he is simply a passionate software developer who cares deeply about the cause that he is espousing. And in fact, I don't really see any issues that the Entity Framework petition mentions that I disagree with. I feel that most of the points made in the petition are valid, or at least valid within a particular situation. The problem that I have is that within the context of the argument so far, it is hard to take the petition at face value. There has been a few loud people in the alt.net community that have attacked people who are pushing (or even just educating people about) the Entity framework.
Here is a quote that Scott Bellware put on Julie Lerman's blog (who has been the recipient of many of these criticisms) when she made a post about how the EF vote of no confidence pointed out problems that they had with their particular development style and then stated "but I just don't have the same needs as those that are listed in the petition".
"I didn't have those needs either, until I learned about how productive I could be over and above what I used to do. From this perspective, after having made a transition to a different perspective, did I see that I always had had those needs, but I just didn't have the perspective that allowed me to see them."
That is wonderful for him, but I think that pretty much sums it up. "You don't think you need this, but you just aren't as experienced as I am, so you don't get it." I'm not sure a lot of people understand that this is the attitude that is turning many people off to the ALT.NET movement. The idea that somehow these people have found the "one true way"® to do software development, and everyone else just clearly hasn't come around yet. Let me just say that no matter how you are developing software, you are wrong. Your methods, practices, patterns, etc... are all wrong. You will never get it right, you will try, and try again, to get it right; but you will always find a better way. You will not find a better way by telling other people that their way sucks, you will only find a better way by trying and experimenting and sharing.
And this is why I am saddened. Not because of the Entity Framework petition, not because of the bashing and name calling, but because I can already start to see the walls going up on either side. Sure, the Entity Framework team responded quickly with word that they were going to meet all of the demands in the petition for the next release. And that is great, but do you think they responded that fast because of the petition? No, of course they didn't, they had already heard the feedback. In fact, this post on the Entity Framework Design Blog, was made at 4:12 PM PST on the 23rd, which oddly enough was about 20 minutes before (by look at the e-mail headers) the original post was made to Yahoo group's.
The ALT.NET movement is currently our only good hope for a more open Microsoft ecosystem, and the progress that has been made up to this point has been amazing. There probably wouldn't even be an asp.net MVC framework coming if it wasn't for the ALT.NET community pushing the envelope so hard with MonoRail. There certainly wouldn't be anywhere near as much excellent information, open discussion, and wonderful projects coming out of the community without ALT.NET. Who would have imagined a few years ago that Microsoft would release the source for the .net framework, or start creating open source projects itself on Codeplex? So keep up the good work ALT.NET community, but if I had to offer any unsolicited advice, it would be to stick with the teaching and stop with the preaching.
Disclaimer:
No ALT.NET members were harmed in the writing of this post.
Posted on 6/24/2008 9:28:48 PM by Justin Etheredge
In my previous post I talked about Linq expression trees and how you can create them in several ways. We talked about how you could create them simply using a lambda expression:
Expression<Func<int, int>> lammy = n => n * 2;
Or how you could create them manually:
ParameterExpression param = Expression.Parameter(typeof (int), "n");
BinaryExpression body =
Expression.Multiply(param, Expression.Constant(2, typeof (int)));
lammy = Expression.Lambda<Func<int, int>>(body, param);
I then showed you some code that would allow us to parse out a string of numbers and math operations and build up an expression tree. Everything looked great, all we had to do was keep parsing our string and chaining the Expressions together using our GetMathExpression method:
public Expression GetMathExpression(char operation,
Expression leftExpression, Expression rightExpression)
{ switch (operation)
{ case '*':
return Expression.MultiplyChecked(leftExpression, rightExpression);
case '+':
return Expression.AddChecked(leftExpression, rightExpression);
case '/':
return Expression.Divide(leftExpression, rightExpression);
case '-':
return Expression.SubtractChecked(leftExpression, rightExpression);
}
return null;
}
So, we just parse through our math operation building up our expression tree. But, as we discussed in the last post, there is a problem. The tree that we built up is going to be in the order that we had in the math equation that we entered. It is totally ignoring order of operations and just going sequentially down the string. So, for a string of "5+2*5+3" we would end up with a tree that looks like this:
Which ends up producing an incorrect answer to our equation, since it would be evaluated as: ((5 + 2) * 5) + 3.
So, what would we have to do in order to get this to evaluate correctly? Well, we would have to manipulate this tree until it took the form of 5 + (2 * 5) + 3. This would create a tree that looks like this:
This way, the 2 * 5 would get evaluated instead of 5 + 2. So you will see that the logic we are employing is simply to start from the top of the tree and start walking down the tree looking for nodes where the order of operations of the child is less then the parent. This is true when we hit the multiply node in the first tree (since add has a lower order of operation than the parent). So, we need to move the + sign up and move the multiplication down.
The steps that you will follow in order to do this is:
- Move the + node in place of the * node, effectively removing the * nodes left node
- Remove the + node's right child and replace it with the * node
- Set the + node's former right child as the left node of the * node
This will move the add above the multiply which will cause it to be evaluated later. So, how do we do this in code?
Well, the first thing we have to do is to be able to walk our expression tree. In this example we are using almost all binary expressions, so it probably wouldn't be too hard to walk this tree, but not all linq expressions have two children. Some have more properties and expressions to them, and it would require quite a bit of code to walk all the different types of expressions. Luckily Microsoft has helped us out with this. In this MSDN page Microsoft shows you how to create an expression visitor, which is just a class that has knowledge of how to walk an expression tree. All of the methods to handle each type of expression are virtual, so you can plug your own logic in as you walk the tree.
Whatever gets returned from each of these methods is used to build up the resulting tree. In this way, by changing what is returned from these methods, we can manipulate the tree. So, first I am going to create a class called OrderOfOperationsVisitor that descends from the ExpressionVisitor class that I pulled from Microsoft.
public class OrderOfOperationsVisitor : ExpressionVisitor
Okay, so we have our class signature and now we just need to overload our method:
protected override Expression VisitBinary(BinaryExpression b)
{ if ((b.Left is BinaryExpression) && IsChildOrderOfOperationsLower(b, b.Left))
{ this.SwapOccurred = true;
return SwapLeft(b, (BinaryExpression)b.Left);
}
if ((b.Right is BinaryExpression) && IsChildOrderOfOperationsLower(b, b.Right))
{ this.SwapOccurred = true;
return SwapRight(b, (BinaryExpression) b.Right);
}
return base.VisitBinary(b);
}
Okay, so this where the magic begins. You can see that we are looking at the children of our BinaryExpression and then testing it to see the order of operations is lower. If it is, then we set a property on our class called "SwapOccurred" to true (remember this for later) and then we swap the current node with that child. If neither node is swapped, then we simply continue on down our tree by calling base.VisitBinary(b) which will simply walk the children of this node.
Since we are only supporting +, -, /, and * our "IsChildOrderOfOperationsLower" method will look like this:
private bool IsChildOrderOfOperationsLower(Expression current, Expression child)
{ if (current.NodeType == ExpressionType.MultiplyChecked
|| current.NodeType == ExpressionType.Divide)
{ if (child.NodeType == ExpressionType.AddChecked ||
child.NodeType == ExpressionType.SubtractChecked)
{ return true;
}
}
return false;
}
And our swap methods look like this:
private Expression SwapLeft(BinaryExpression expression, BinaryExpression left)
{ Expression right = Expression.MakeBinary(expression.NodeType, left.Right, expression.Right);
return Expression.MakeBinary(left.NodeType, left.Left, right);
}
private Expression SwapRight(BinaryExpression expression, BinaryExpression right)
{ Expression left = Expression.MakeBinary(expression.NodeType, expression.Left, right.Left);
return Expression.MakeBinary(right.NodeType, left, right.Right);
}
Easy enough to see what is happening there. I am creating new expressions to do exactly the process that we outlined above for swapping a node with its parent.
Now we just have to use this class on our newly constructed tree:
var orderOfOperationsVisitor = new OrderOfOperationsVisitor();
mathExpression = orderOfOperationsVisitor.Visit(mathExpression);
while (orderOfOperationsVisitor.SwapOccurred)
{ orderOfOperationsVisitor = new OrderOfOperationsVisitor();
mathExpression = orderOfOperationsVisitor.Visit(mathExpression);
}
Notice that we run it once, then we start looping on "SwapOccurred". This is because after we reorder the nodes, we simply return the nodes back from our method and stop walking the tree there. We need to check below that node in the tree to make sure that there isn't anything below it that needs to be swapped. Another issue is that a node may need to be moved up multiple levels. Like this:
Since we can't walk the tree backwards, we have to make multiple passes in order to get the '+' node to the top.
So, now that we have done all of this, we can look at our expression tree before:
and after:
Success!
Well, I hope that you have found this little two part series on expression trees to be useful. I have attached the source for this project here, so feel free to download it and play around with it!
Posted on 6/23/2008 2:56:00 PM by Justin Etheredge
(Part 2 has been posted)
This Thursday we (as in the Richmond VA community) are having a Meet and Code dinner where we are going to be talking about Linq Expression trees. A while back I made a post about dynamically generating Lambdas, and since the topic of the dinner this Thursday is Expression trees, I wanted to throw together a demo that uses dynamically constructed expression trees to do something useful. So, if you are attending this Thursday, then you may want to skip this post so that you'll have something new to see this Thursday!
In case you don't know what Linq Expression Trees are, they are ASTs (Abstract Syntax Trees) that can be built out of lambdas or Linq expressions. If you want to get an expression tree out of a lambda then you can simply wrap the Func delegate in the generic Expression class like this:
Expression<Func<int, int>> lammy = n => n * 2;
Interesting isn't it? Yep, the C# compiler actually generates two different things based on the type you are assigning to! (Did you know that MSIL supports overloading by return type?) So, if I were to do this, it would generate the actual delegate:
Func<int, int> lammy = n => n * 2;
Kinda cool, but that isn't what we are talking about here. What we are here for is to learn how these suckers work. So, what does the expression tree from the above Expression look like?
Now you can see why it is called an abstract syntax tree. I removed the Lambda expression that this is wrapped in, but this is the important part. Here you see that we have a BinaryExpression of type Multiply (or CheckedMultiply) with a left and right node that represent the parameter to the lambda and the constant value of two.
Now, these syntax trees come in handy for a variety of reasons (there are probably a few more):
- Tree modification before execution - You can actually modify the tree before it is compiled. You could join operations or simplify the tree before compiling it.
- Combining trees: This is in line with number 1, but you can take multiple expression trees and combine them using new Expression tree nodes. (Yes, you can manually generate these, and we will get into those later on)
- Alternate Execution: You can take an expression tree and interpret it and execute it in your own manner. This is precisely what Linq To Sql does. It takes an expression tree and interprets it, then turns it into SQL instead of runnable MSIL.
So, in this post I am going to look at 1 and 2. But first lets look at how we would build up an expression tree manually. In order to build the above expression manually the code would look like this:
ParameterExpression param = Expression.Parameter(typeof (int), "n");
BinaryExpression body =
Expression.Multiply(param, Expression.Constant(2, typeof (int)));
lammy = Expression.Lambda<Func<int, int>>(body, param);
Not too hard to follow. We declare our parameter and then we declare our lambda body as an multiply BinaryExpression with the left op being the param and the right op being the constant value 2. We can then create our lambda expression passing in our body and our parameters. In order to use this lambda, all we have to do is call:
Func<int,int> delly = lammy.Compile();
And now we have a usable delegate that we can call. Pretty sweet, but you are probably thinking, why would I want to go through all that when I can just say:
Func<int, int> delly = n => n * 2;
And you'd be right, if you already knew what the lambda was going to be. But what if you don't know what the lambda is going to be? Then how would you define it in that nice little syntax. Well, you wouldn't, you'd have to build it up yourself. But the good news is that it is very possible.
So, lets say that you are building a calculator program. Wow, I just happened to have built one as an example! That is amazing! And so I want to be able to enter a string of math operations like this:
Well, we could walk from left to right taking each number and math operation building up a tree as we go. The result would end up looking something like this:
If you have ever had to do much tree traversal, you will notice that this tree will most likely be evaluated using a depth first in-order traversal. So what will happen is that we start at the top with the '+' and we will go to the left child, and then when we get there we will go to its left child until we run out of left children. So, we will hit the 5 and then pop back up to the '+' and then we will go for the right child which is '2'. So, 5 + 2 is evaluated to 7 and then 7 * 5 is evaluated to 35 and then we add 35 + 3 to get 38. We just keep falling back up the tree going left, center, right. The code used to get here looks something like this (several method calls are factored out, but they have meaningful names):
string currentDigit = string.Empty;
char currentOperation = ' ';
foreach (char c in equation)
{ if (Char.IsDigit(c))
{
currentDigit += c;
}
else
{
if (!String.IsNullOrEmpty(currentDigit))
{ if (leftExpression == null)
{
leftExpression = GetDigitConstant(currentDigit);
}
else
{
rightExpression = GetDigitConstant(currentDigit);
mathExpression =
GetMathExpression(currentOperation, leftExpression, rightExpression);
leftExpression = mathExpression;
}
currentDigit = string.Empty;
}
currentOperation = c;
}
}
rightExpression = GetDigitConstant(currentDigit);
mathExpression = GetMathExpression(currentOperation, leftExpression, rightExpression);
Func<int> func = CompileExpression(mathExpression);
The only problem with this is that it is wrooooooooooooooong (and it is pretty ugly)! Ever heard of a little thing called order of operations?
If, by some small miracle, you have not heard of order of operations I am going to simplify it for you real quick. Since my app doesn't support parenthesis, and only basic math functions such as *, /, +, - we are just going to say that you do all * and / before + and -. So, if you had 5+3*6 the answer would be 23 and not 48.
If we wanted to get our math expression into order of operations, how would we do it? Would we do several passes through our expression string and try to build up our multiplies and divides first and then do some fancy stuff in order to get it all in order? Or could we just build our tree and then manipulate it in tree form? Hmmmmmmm. Well, when dealing with trees there are tons well known manipulations you can do in order to the tree in the form you want, so I think I am going to go that route....
Sorry, it is getting pretty late, I think I am going to stop there for the evening. In the next installment I am going to talk about the expression visitor class and how we can use it to correct our order of operations problem. I will also attach the source code to the entire project in the next post. I hope this helps!
Posted on 6/23/2008 12:37:01 AM by Justin Etheredge
Well, it looks like I am going to CodeStock. In case you don't know, it is a one day developer conference in Knoxville Tennessee. The list of sessions looks awesome, and is only eclipsed by the list of experts that are going to be there. The date is August 8th, so get it on your calendar!
It is going to be quite the long trip from Richmond (about 7 hours according to Google Maps), but I will be riding with Kevin Hazzard so the trip should go by pretty quick. The trip is a loooong drive down 81 (hopefully Kevin has some reliable transportation!), or if we carpool with some of the guys out of Raleigh then it'll be a long drive down 40.
Hopefully if you live in the mid-Atlantic region then you will be able to get out there!
Posted on 6/20/2008 3:11:00 PM by Justin Etheredge
The June Richmond Meet and Code dinner has been scheduled for June 26th! We are being graciously hosted again by SnagAJob.com. They have a wonderful meeting room setup where everyone can sit and eat and watch a presentation or have a good argument discussion.
This month's topic is going to be Linq expression trees. I will have a little demo setup to show off expression trees, but I hope that others will come armed with something to share. The floor is also open to discussing any topics or asking any questions, so if you need help with something please don't hesitate to ask.
There will also be food provided, so come hungry!
Click here to signup so that we know how many are coming.