tag:blogger.com,1999:blog-68497606236097713632024-03-12T19:11:26.854-07:00Tim KelloggTinkering with ideas. Code, language, software engineeringAnonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.comBlogger47125tag:blogger.com,1999:blog-6849760623609771363.post-72751374198411335622012-04-22T10:40:00.002-07:002012-04-22T10:40:52.331-07:00Blog is movingThanks to everyone who reads my blog. Recently I made the move and switched to using Github pages for my blog. I also <a href="http://timkellogg.me/" target="_blank">bought a domain name</a> that sounds kind of cool. Now I have a place where I can add other content besides blog posts.<br />
<br />
I made the switch because now I can write blog posts in markdown on Vim, and offline so I can blog while riding the bus or on an airplane. I also found that writing code in Blogger was a huge pain, but <a href="https://github.com/mojombo/jekyll" target="_blank">jekyll</a> provides inline code highlighting for a large number of syntaxes. After considering it for a while, I realized the benefits of switching far outweighed the features blogger offers.<br />
<br />
I've already written some <a href="http://info.timkellogg.me/blog/2012/03/24/why-object-ids-primary-keys-are" target="_blank">pretty</a> <a href="http://info.timkellogg.me/blog/2012/04/18/code-coverage-metrics" target="_blank">interesting</a> <a href="http://info.timkellogg.me/blog/2012/04/22/why-open-source-is-worth-your-time/" target="_blank">posts</a>. I have a lot more planned, so please keep with me. If you follow this blog via the atom feed, I also have <a href="http://info.timkellogg.me/blog/atom.xml" target="_blank">a new atom feed</a>, so please update your links!Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com1tag:blogger.com,1999:blog-6849760623609771363.post-57692668783347282302012-03-24T14:07:00.000-07:002012-03-24T14:07:25.481-07:00Why Object IDs & Primary Keys Are Implementation DetailsRecently <a href="http://blog.timkellogg.me/2012/03/abstract-data-layer-part-1-object-id.html" target="_blank">I wrote a post</a> about a project that I was working on with an abstracted data layer concept that can work in the context of either relational or document data store. In retrospect I think I brushed too quickly over the details of why I think object identifiers (and primary keys) are a part of the implementation that should be hidden, when possible. To explain what I mean I'll use a surreal-world story.<br />
<br />
<span style="font-size: large;">The Situation</span><br />
<br />
You are the chief software engineer at a software company. One day your product manager comes to you with a list of ideas for a new product where users can post definitions to slang words, like a dictionary. He says people are going to love this new app because everyone has a different idea of what words mean. After talking with him to establish ubiquitous language and identify nouns and verbs, you crank up some <a href="https://twitter.com/#!/search/%23codingmusic" target="_blank">coding music</a> and hack out some model classes.<br />
<br />
<script src="https://gist.github.com/2166519.js?file=Word-Definition.cs">
</script>
<br />
A weekend later you finish coding the app using Int32s (<span style="font-family: 'Courier New', Courier, monospace;">int</span>) as the identity data type for most of your models because it's usually big enough and works well as a primary key. Honestly, you didn't really think about it because its what you always do.<br />
<br />
After the launch your app quickly gains popularity with the user base doubling every day. Not only that, but as more definitions get posted, more people are attracted to the site and post their own word definitions. While reviewing the exponential data growth figures, your DBA decides that <span style="font-family: 'Courier New', Courier, monospace;">Definition.Id</span> should be changed to an Int64 (<span style="font-family: 'Courier New', Courier, monospace;">long</span>) to accommodate the rapidly multiplying postings.<br />
<br />
Let's stop for a minute and review what the <i>business needs</i> were. Your product manager wants an app where people can post words and definitions. Each word has many definitions. There's no talk in the business domain of tables and primary keys. But you included those concepts in the model anyway, because that's how you think about your data.<br />
<br />
The DBA chose to make the ID into a larger number to accommodate a larger amount of data. So now to help optimize the database, you are forced to update all your <i>business logic</i> to work nicely with the <i>data logic</i>.<br />
<br />
<span style="font-size: large;">Data Logic Was Meant to Live in the Database</span><br />
<br />
The trouble with tying data logic closely to business logic is that the database isn't part of your business plan. As your application grows you'll have to tweak your database to squeeze out performance - or even swap it out for <a href="http://cassandra.apache.org/" target="_blank">Cassandra</a>. Databases are good at data logic because they are declarative. You can usually tune performance without affecting how the data is worked with. When you place an index, it doesn't affect how you write a SELECT or UPDATE statement, just how fast it runs.<br />
<br />
At the same time, databases are also very procedural things. When you put business logic in stored procedures you lose the benefits of object oriented programming. It also makes unit tests complicated, slow, and fragile (which is why most people don't unit test the database). In the end, it's best to let your database optimize how data is stored and retrieved and keep your domain models clean and focused on the business needs.<br />
<br />
<span style="font-size: large;">The Type of the Object ID Is an Implementation Detail</span><br />
<br />
Lets say you hire a new COO that lives in Silicon Valley and thinks the latest coolest technology is always the gateway to success. With the new growth he decides that you should rewrite the dictionary application to use <a href="http://www.mongodb.org/display/DOCS/Introduction" target="_blank">MongoDB</a> because it's the only way your application can scale to meet the needs of the business. While evaluating Mongo you draw out what an example word and definitions might look like when stored as <a href="http://bsonspec.org/" target="_blank">BSON</a>:<br />
<br />
<script src="https://gist.github.com/2166519.js?file=Word-definitions.json">
</script>
<br />
In Mongo, <a href="http://www.mongodb.org/display/DOCS/Schema+Design#SchemaDesign-EmbeddingandLinking" target="_blank">you usually would store the Definitions inline with the Word</a>. Now there is no need for a Definition.Id or Definition.WordId because all of this is implicit. Not only that, but Word.Id is now an <a href="http://www.mongodb.org/display/DOCS/Object+IDs" target="_blank">ObjectId</a> - a very different 12 byte number that includes time and sequence components. In order to update your application to work with Mongo, you'll have to update all references IDs to use these ObjectIds.<br />
<br />
The ID is an implementation concern. In a centralized SQL database, sequential integers make sense. In a distributed environment like Mongo, ObjectIDs offer more advantages. Either way, the type of your ID is an implementation detail.<br />
<br />
<span style="font-size: large;">Encapsulation Requires That You Hide Implementation Details</span><br />
<br />
Most OO programmers understand that encapsulation means that an object <i>has</i> or <i>contains</i> another object. However, some forget that a <a href="http://en.wikipedia.org/wiki/Encapsulation_(object-oriented_programming)" target="_blank">large part of encapsulation</a> is that you should keep the <a href="http://stackoverflow.com/a/1777728/503826" target="_blank">implementation details</a> of an object hidden from other objects. When the details of an object leak into other objects, the contract is broken and you <a href="http://www.joelonsoftware.com/articles/LeakyAbstractions.html" target="_blank">lose the benefits of the OO abstraction</a>.<br />
<br />
Any ORM tool should give you the ability to select protected (if not private) members of the object to be persisted. If it doesn't, it's not using because it'll cause too great of a compromise in design. This is how we should have been allowed to write our objects from the start:<br />
<br />
<script src="https://gist.github.com/2166519.js?file=Word-definitions-correct.cs">
</script>
<br />
<span style="font-size: large;">But Dynamic Languages Diffuse The Problem</span><br />
<br />
If you're in a dynamic language like Ruby or Node.js this is less of an issue. Most of my argument hinges on the idea that your API will latch onto the object's ID and insist that all methods that use it will match. This is really just a constraint of strict statically typed languages. Even implicit typing will mitigate the issue some.<br />
<br />
You can notice above that I got around the constraint by using <span style="font-family: 'Courier New', Courier, monospace;">object</span> as the ID type. This is really what you want. It's telling the compiler and API that you really, shouldn't care what the type is - it's an implementation detail. You shouldn't run into many problems as long as you are keeping the ID properly encapsulated within the object.<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-42558838534218515982012-03-19T20:56:00.000-07:002012-03-19T21:00:22.565-07:00Abstract Data Layer Part 1: Object ID Types And ConventionsIn February I went to the MongoDB conference in Boulder. That day was my first real taste of any sort of document oriented database. Since then I've played around with Mongo in C#, Node.JS and natively in the Mongo shell. Since then, I also can't help feeling overwhelmingly happy when thinking about how I can use Mongo for a project.<br />
<br />
At Alteryx we're entering a project where we require some specific business needs. We require an extremely fast and scalable database, hence Mongo. But we also need to package our product for on-premise installations, which I hear requires that we also support certain SQL databases.<br />
<br />
<i>...I don't actually understand why enterprises insist on using SQL. I'm told that enterprise DBA's want control over everything, and they don't want to learn new products like MongoDB. To me, it seems that 3rd products that are bought would be exempt from DBA optimizations & other meddling. But I guess I wouldn't know what it takes to be an enterprise DBA, so I'll shut up about this now. Just my thoughts...</i><br />
<div>
<br />
Since relational databases are a lot different than document oriented databases I decided to use NHibernate as an ORM since they've already figured out a lot of the hard problems. I chose NHibernate over Entity Framework mainly because I already know NHibernate, and I know that it has good support across many databases. Nothing against EF in particular.<br />
<br />
I've been working on this for a week or so. I've gotten pretty deep into the details so I thought a blog post would be a good way to step out and think about what I've done and where I'm going. The design is mostly mine (of course, I stand on the backs of giants) and really just ties together robust frameworks.<br />
<br />
<span style="font-size: x-large;">Convention Based Object Model</span><br />
<br />
In order to remain agnostic toward relational/document structure, I decided that there would have to be some basic assumptions or maxims. I like the idea of convention-based frameworks and I really think its the best way to go about building this kind of infrastructure. Also, conventions are a great way to enforce assumptions and keep things simple.<br />
<br />
<span style="font-size: large;">IDs Are Platform Dependent</span><br />
<br />
It's not something I really thought about before this. In relational databases we'll often use an integer as the object ID. They're nice because they're small, simple, and sequential. However, Mongo assumes that you want to be extremely distributed. Dense sequential IDs (like int identity) run into all kinds of race conditions and collisions in distributed environments (unless you choose a master ID-assigner, which kind of ruins the point of being distributed).<br />
<br />
MongoDB uses <a href="http://www.mongodb.org/display/DOCS/Object+IDs" target="_blank">a very long (12 byte) semi-sequential number</a>. It's semi-sequential in that every new ID is a bigger number than the IDs generated before it, but not necessarily just +1. Regardless, it's impractical to use regular integers in Mongo and also a little impractical to use long semi-sequential numbers in SQL.<br />
<br />
As a result, I chose to use <span style="font-family: 'Courier New', Courier, monospace;">System.Object</span> as the ID type for all identifiers. NHibernate can be configured to use objects as integers with native auto-increment after some tweaking. The Mongo C# driver also supports object IDs with client-side assignment.<br />
<br />
Ideally, I would like to write some sort of <span style="font-family: 'Courier New', Courier, monospace;">IdType</span> struct that contains an enumeration and object value (I'm thinking along the lines of a discriminated union here). This would help make IDs be more distinctive and easier to attach extension methods or additional APIs. I'd also like to make IDs protected by default (instead of public).<br />
<br />
<span style="font-size: large;">The Domain Object</span><br />
<br />
I also created a root object for all persistent objects to derive from. This is a fairly common pattern, especially in frameworks where there is a lot of generic or meta-programming.<br />
<br />
<script src="https://gist.github.com/2130909.js?file=DomainObject-simple.cs">
</script>
<br />
I had <span style="font-family: 'Courier New', Courier, monospace;">DomainObject</span> implement an <span style="font-family: 'Courier New', Courier, monospace;">IDomainObject</span> interface so that in all my meta-programming I can refer to <span style="font-family: 'Courier New', Courier, monospace;">IDomainObject</span>. That way there shouldn't ever be a corner case where we can't or shouldn't descend from <span style="font-family: 'Courier New', Courier, monospace;">DomainObject</span> but have to anyway (separate implementation from interface).<br />
<br />
<script src="https://gist.github.com/2130909.js?file=User-Name.cs">
</script>
<br />
The User and Name objects are simple, as you can expect any NHibernate object model to look like. The idea is to keep them simple and keep business and data logic elsewhere.<br />
<br />
<span style="font-size: large;">Are You Interested?</span><br />
<br />
From what I can tell, I think we're breaking ground on this project. It doesn't seem like too many people have tried to make a framework to support both relational and document data stores. Initially I was hesitant to support both relational and document stores. But I think there are some excellent side effects that I will outline in upcoming posts.<br />
<br />
The content I've written about so far is only a small fraction of what it took to get this on it's feet. Someone once said that <a href="http://tom.preston-werner.com/2011/11/22/open-source-everything.html" target="_blank">you should open source (almost) everything</a>. So, if you (or anyone you know) would like to see the full uncensored code for this, let me know so I can start corporate conversations in that direction. </div>Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com3tag:blogger.com,1999:blog-6849760623609771363.post-49218645707462114722012-03-10T12:25:00.001-08:002012-03-10T12:25:24.337-08:00Discriminated Unions in C# Mono CompilerRecently I've been using F# a bit. F# is .NET's functional language (the syntax of F# 1.0 was backward compatible with OCaml, but 2.0 has diverged enough to make it more distinct). Learning F# was a huge mind-shift from the C-family of languages. Of all the features of F#, like implicit typing, tail recursion, and monads, many people list discriminated unions as their favorite.<br />
<br />
Discriminated unions feel like C# enums on the surface. For instance, a union that can represent states of a light switch:<br />
<br />
<script src="https://gist.github.com/2012928.js?file=union.fs">
</script><br />
<br />
This example is really no different from C# enums. Discriminated unions, however, can hold data. For instance, consider when our light switch needs to also be a dimmer:<br />
<br />
<script src="https://gist.github.com/2012928.js?file=discriminated-union.fs">
</script><br />
<br />
In C# we would have had to rewrite this whole program to handle the new dimmer requirement. Instead, we can just tack on a new state that holds data.<br />
<br />
When you're deep in the F# mindset, this structure makes perfect sense. But try implementing a discriminated union in C#. There's the enum-like part, but there's also the part that holds different sizes of data. There's <a href="http://stackoverflow.com/a/2321922/503826" target="_blank">a great stackoverflow answer</a> that explains how the F# compiler handles discriminated unions internally. It requires 1 enum, 1 abstract class and <i>n</i> concrete implementations of the abstract class. It's quite over-complicated to use in every-day C#.<br />
<br />
Nevertheless, I really want to use discriminated unions in my C# code because of how easy they make state machines & workflows. I've been brainstorming how to do this. There are several implementations as C# 3.5 libraries, but they're cumbersome to use. I've been looking at the source code for the mono C# compiler, and I think I want to go the route of forking the compiler for a proof-of-concept.<br />
<br />
I'm debating what the syntax should be. I figure that the change would be easier if I re-used existing constructs and just tweaked them to work with the new concepts.<br />
<br />
<script src="https://gist.github.com/2012928.js?file=case-block.cs">
</script><br />
<br />
I've been debating if the Dimmed case should retain the regular case syntax or get a lambda-like syntax:<br />
<br />
<script src="https://gist.github.com/2012928.js?file=case-lambda.cs">
</script><br />
<br />
I'm leaning toward the lambda syntax due to how C# usually handles variable scope. I've barely just cloned the mono repository and started reading the design documents to orient myself with the compiler. This could be a huge project, so I'm not sure how far I'll actually get. But this is a very interesting idea that I want to try hashing out.<br />
<br />Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-11828243317507958372012-02-29T07:16:00.000-08:002012-02-29T07:18:43.228-08:00One Thing I Learned From F# (Nulls Are Bad)Recently I started contributing to <a href="https://github.com/jaredpar/VsVim" target="_blank">VsVim</a>, a Visual Studio plugin that emulates Vim. When he was starting the project, Jared Parsons decided to write the bulk of it in F#. He did this mostly as a chance to learn a new language but also because it's a solid first class alternative to C#. For instance, F#'s features like pattern matching and discriminated unions are a natural fit for state machines like Vim.<br />
<br />
This is my first experience with a truly functional language. For those who aren't familiar with F#, it's essentially OCaml.NET (the <a href="http://en.wikibooks.org/wiki/F_Sharp_Programming" target="_blank">F# book</a> uses OCaml for it's markup syntax), but also draws roots from Haskell. It's a big mind shift from imperative and pure object oriented languages, but one I'd definitely recommend to any developer who wants to be better.<br />
<br />
Since I've been working on VsVim, I've been using F# in my spare time but C# in my regular day job. The longer I use F# the more I want C# to do what F# does. The biggest example is how F# handles nulls.<br />
<br />
In C# (and Ruby, Python, and any imperative language) most values can be null, and null is a natural state for a variable to be in. In fact (partly due to SQL), null is used whenever a value is empty or doesn't exist yet. In C# and Java, null is the default value for any member reference, you don't even need to explicitly initialize it. As a result, you often end up with a lot of null pointer exceptions due to sloppy programming. After all, it's kind of hard to remember to check for null every time you use a variable.<br />
<br />
In F#, nothing is null (that's not entirely true, but in it's natural state it's true enough). Typically you'll use options instead of null. For instance, if you have a function that fails to find or calculate something you might return null in imperative languages (and the actual value if successful). However, in F# you use an option type and return None on failure and Some value on success.<br />
<br />
<script src="https://gist.github.com/1941345.js">
</script>
<br />
Here, every time you call find(kittens) you get back an option type. This type isn't a string, so you can't just start using string methods and get a null pointer exception. Instead, you have to extract the string value from the option type before it can be used.<br />
<br />
At this point you might be thinking, "why would I want to do that? It looks like a lot of extra code". However, I challenge you to find a crashing bug in VsVim. Every time we have an instance of an invalid state we are forced to deal with it on the spot. Every invalid state is dealt with in a way that makes sense.<br />
<br />
If we wrote it in C# it would be incredibly easy to get lazy while working late at night and forget to check for null and cause the plugin to crash. Instead, the only bugs we have are behavior quirks. If we ever have a crashing bug, the chances are the null value originated in C# code from Visual Studio or the .NET Framework and we forgot to check.<br />
<br />
<i><a href="http://news.ycombinator.com/item?id=3648104" target="_blank">Discussion on HN</a></i>Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com2tag:blogger.com,1999:blog-6849760623609771363.post-74785641341043917692012-02-10T06:16:00.000-08:002012-02-10T06:16:26.594-08:00C# Reflection Performance And RubyI've always known that reflection method invocations C# are slower than regular invocations, but I've never never known to what extent. So I set out to make an experiment to demonstrate the performance of several ways to invoke a method. Frameworks like <a href="http://nhforge.org/" target="_blank">NHibernate</a> or the <a href="http://www.mongodb.org/display/DOCS/CSharp+Language+Center" target="_blank">mongoDB driver</a> are known to serialize and deserialize objects. In order to do either of these activities they have to scan the properties of an object and dynamically invoke them to get or set the values. Normally this is done via reflection. However, I want to know if the possibility of <a href="http://en.wikipedia.org/wiki/Memoization" target="_blank">memoizing</a> a method call as an expression tree or delegate could offer significant performance benefits. On the side, I also want to see how C# reflection compares to Ruby method invocations.<br />
<br />
I posted the full source to <a href="https://github.com/tkellogg/ReflectionPropertyInvokeBenchmark">a public github repo</a>. To quickly summarize, I wrote code that sets a property on an object 100 million times in a loop. Any setup (like finding a <span style="font-family: 'Courier New', Courier, monospace;">PropertyInfo</span> or <span style="font-family: 'Courier New', Courier, monospace;">MethodInfo</span>) is not included in the timings. I also checked the generated IL to make sure the compiler wasn't optimizing the loops. Please browse the code there if you need the gritty details.<br />
<br />
Before I get into the implementation details, here are the results:<br />
<br />
<iframe frameborder="no" height="300px" scrolling="no" src="http://www.google.com/fusiontables/embedviz?&containerId=gviz_canvas&q=select+col0%2C+col1+from+2840399+&qrs=where+col0+%3E%3D+&qre=+and+col0+%3C%3D+&qe=+limit+6&viz=GVIZ&t=BAR&width=500&height=300" width="500px"></iframe>
<br />
<br />
You can see that a reflection invoke is on the order of a hundred times slower than a normal property (set) invocation.<br />
<br />
Here's the same chart but without the reflection invocation. It does a better job of showing the scale between the other tests.<br />
<br />
<iframe frameborder="no" height="300px" scrolling="no" src="http://www.google.com/fusiontables/embedviz?&containerId=gviz_canvas&q=select+col0%2C+col1+from+2840399+where+col1+%3C+'25000'&qrs=+and+col0+%3E%3D+&qre=+and+col0+%3C%3D+&qe=+limit+5&viz=GVIZ&t=BAR&width=500&height=300" width="500px"></iframe>
<br />
<br />
Obviously, the lesson here is to directly invoke methods and properties when possible. However, there are times when you don't know what a type looks like at compile time. Again, object serialization/deserialization would be one of those use cases.<br />
<br />
Here's an explanation of each of the tests:<br />
<br />
<span style="font-size: large;">Reflection Invoke</span> (<a href="https://github.com/tkellogg/ReflectionPropertyInvokeBenchmark/blob/363b1242a0210c9d7deb4db2571134333476e96b/ReflectionBenchmark/Program.cs#L59" target="_blank">link</a>)<br />
<br />
This is essentially <span style="font-family: 'Courier New', Courier, monospace;">methodInfo.Invoke(obj, new[]{ value }</span> on the setter method of the property. It is by far the slowest approach to the problem. It's also the most common way to solve the problem of insufficient pre-compile time knowledge.<br />
<br />
<span style="font-size: large;">Direct Invoke</span> (<a href="https://github.com/tkellogg/ReflectionPropertyInvokeBenchmark/blob/363b1242a0210c9d7deb4db2571134333476e96b/ReflectionBenchmark/Program.cs#L47" target="_blank">link</a>)<br />
<br />
This is nothing other than <span style="font-family: 'Courier New', Courier, monospace;">obj.Property = value</span>. Its as fast as it gets, but impractical for use cases where you don't have pre-compile time knowledge of the type.<br />
<br />
<span style="font-size: large;">Closure</span> (<a href="https://github.com/tkellogg/ReflectionPropertyInvokeBenchmark/blob/363b1242a0210c9d7deb4db2571134333476e96b/ReflectionBenchmark/Program.cs#L92" target="_blank">link</a>)<br />
<br />
This isn't much more flexible than a direct invoke, but I thought it would be interesting to see how the performance degraded. This is where you create a function/closure ( <span style="font-family: 'Courier New', Courier, monospace;"><exampletype, string="">(x,y) => x.Property = y</exampletype,></span>) prior to the loop and just invoke the function inside the loop (<span style="font-family: 'Courier New', Courier, monospace;">action(obj, value)</span>). At first sight it appears to be half as fast as a direct invoke, but there are actually two method calls involved here, so it's actually not any slower than a direct invoke.<br />
<br />
<span style="font-size: large;">Dynamic Dispatch</span> (<a href="https://github.com/tkellogg/ReflectionPropertyInvokeBenchmark/blob/363b1242a0210c9d7deb4db2571134333476e96b/ReflectionBenchmark/Program.cs#L78" target="_blank">link</a>)<br />
<br />
This uses the C# 4.0 dynamic feature directly. To do this, I declared the variable as dynamic and assigned it using the same syntax as a direct invoke. Interestingly, this performs only 6x slower than direct invoke and about 20x faster than reflection invoke. Take note, if you need reflection, use dynamic as often as possible since it can really speed up method invocation.<br />
<br />
<span style="font-size: large;">Expression Tree</span> (<a href="https://github.com/tkellogg/ReflectionPropertyInvokeBenchmark/blob/363b1242a0210c9d7deb4db2571134333476e96b/ReflectionBenchmark/Program.cs#L110" target="_blank">link</a>)<br />
<br />
The shortcoming of most of the previous approaches is that they require pre-compile time knowledge of the type. This time I tried building an expression tree (a C# 3.0 feature) and compiled a delegate that invokes the setter. This makes it flexible enough that you can call any property of an object without compile-time knowledge of the name, as long as you know the return type. In this example, like the closure, we're indirectly setting the property, so two method calls. With this in mind, it took almost 2.5 times as long as the closure example, even though they should be functionally equivalent operations. It must be that expression trees compiled to delegates aren't actually as simple as they appear.<br />
<br />
<span style="font-size: large;">Expression Tree with Dynamic Dispatch</span> (<a href="https://github.com/tkellogg/ReflectionPropertyInvokeBenchmark/blob/363b1242a0210c9d7deb4db2571134333476e96b/ReflectionBenchmark/Program.cs#L141" target="_blank">link</a>)<br />
<br />
Since the expression tree approach requires compile-time knowledge of the return type, it isn't as flexible. Ideally you could use C# 4.0's covariance feature and cast it to <span style="font-family: 'Courier New', Courier, monospace;">Action<object, object=""><object, object=""></object,></object,></span> which compiles, but fails at runtime. So for this one, I just assigned the closure to a variable typed as <span style="font-family: 'Courier New', Courier, monospace;">dynamic</span> to get around the compile/runtime casting issues.<br />
<br />
As expected, it's the slowest approach. However, its still 16 times faster than direct reflection. Perhaps, memoizing method calls, like property sets and gets, like this would actually yield a significant performance improvement.<br />
<br />
<span style="font-size: x-large;">Compared To Ruby</span><br />
<br />
I thought I'd compare these results to Ruby where all method calls are dynamic. In Ruby, a method call looks first in the object's immediate class and then climbs the ladder of parent classes until it finds a suitable method to invoke. Because of this behavior I thought I would be interesting to also try a worst-case scenario with a deep level of inheritance.<br />
<br />
To do this fairly, I initially wrote a <span style="font-family: 'Courier New', Courier, monospace;">while</span> loop in Ruby that counted to 100 million. I rewrote the while loop in <span style="font-family: 'Courier New', Courier, monospace;">n.each</span> syntax and saw the execution time get cut in half. Since I'm really just trying to measure method invocation time, I stuck with the <span style="font-family: 'Courier New', Courier, monospace;">n.each</span> syntax.<br />
<br />
<iframe frameborder="no" height="300px" scrolling="no" src="https://www.google.com/fusiontables/embedviz?&containerId=gviz_canvas&q=select+col0%2C+col1+from+2846447+&qrs=where+col0+%3E%3D+&qre=+and+col0+%3C%3D+&qe=+limit+4&viz=GVIZ&t=BAR&width=500&height=300" width="500px"></iframe>
<br />
<br />
I honestly thought C# Reflection would be significantly faster than the Ruby with 5 layers of in inheritance. While C# already holds a reference to the method (<span style="font-family: 'Courier New', Courier, monospace;">MethodInfo</span>), Ruby has to search up the ladder for the method each time. I suppose Ruby's performance could be due to the fact that it's written in C and specializes in dynamic method invocation.<br />
<br />
Also, it interests me why C# dynamic is so much faster than Ruby or reflection. I took a look at the IL code where the dynamic invoke was happening and was surprised to find a <span style="font-family: 'Courier New', Courier, monospace;">callvirt</span> instruction. I guess I was expecting some sort of specialized <span style="font-family: 'Courier New', Courier, monospace;">calldynamic</span> instruction (<a href="http://java.sun.com/developer/technicalArticles/DynTypeLang/" target="_blank">Java 7 has one</a>). The answer is actually a little more complicated. There seems to be several calls - most are <span style="font-family: 'Courier New', Courier, monospace;">call</span> instructions to set the stage (<span style="font-family: 'Courier New', Courier, monospace;">CSharpArgumentInfo.Create</span>) and one <span style="font-family: 'Courier New', Courier, monospace;">callvirt</span> instruction to actually invoke the method.<br />
<br />
<span style="font-size: x-large;">Conclusion</span><br />
<br />
Since the trend of C# is going towards using more <a href="http://msdn.microsoft.com/en-us/library/bb397947.aspx" target="_blank">Linq</a>, I find it interesting how much of a performance hit developers are willing to exchange for more readable and compact code. In the grand scheme of things, the performance of even a slow reflection invoke is probably insignificant compared to other bottlenecks like database, HTTP, filesystem, etc.<br />
<br />
It seems that I've proved the point that I set out to prove. There is quite a bit of performance to be gained by memoizing method calls into expression trees. The application would obviously be best in JSON serialization, ORM, or anywhere when you have to get/set lots of properties on an object with no compile-time knowledge of the type. Very few people, if any, are doing this - probably because of the added complexity. The next step will be to (hopefully) build a working prototype.<br />
<br />
<br />Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com6tag:blogger.com,1999:blog-6849760623609771363.post-80320487430277466332012-02-03T20:17:00.000-08:002012-02-03T20:17:25.462-08:00Thoughts on the C# driver for MongoDBI recently started a new job with a software company in Boulder. Our project this year is rewriting the existing product (not a clean rewrite, more like rewrite & evolve). One of the changes we're making is using <a href="http://www.mongodb.org/">MongoDB</a> instead of <a href="http://en.wikipedia.org/wiki/Transact-SQL">T-SQL</a>. Since we're going to be investing pretty heavily in Mongo we all attended the mongo conference in Boulder on Wednesday. The information was great and now I'm ready to dig into my first app. Today I played around with some test code and made some notes about features/shortcomings of the <a href="http://www.mongodb.org/display/DOCS/CSharp+Driver+Tutorial#CSharpDriverTutorial-Introduction">C# driver</a>.<br />
<br />
First of all, the so-called "driver" is much full featured than a typical SQL driver. It includes features to map documents directly to CLR objects (from here on I'll just say <i>document</i> if I mean Mongo BSON document and <i>object</i> for CLR object). There's plans to support Linq directly from the driver. So right off I'm impressed with the richness of the driver. However, I noticed some shortcomings.<br />
<br />
For instance, all properties in the document must be present (and of the right type) in the object. I perceived this as a shortcoming because this is unlike regular JSON serialization where missing properties are ignored. After thinking a little further, this is probably what most C# developers would want since the behavior caters toward strongly typed languages that prefer fail-fast behavior. If you know a particular document might have extraneous properties that aren't in the object, you can use the <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">BsonIgnoreExtraElements</span> attribute.<br />
<br />
Thinking about this behavior, refactor renaming properties could be less trivial. You would have to run a data migration script to rename the property (mongo does have <a href="http://www.mongodb.org/display/DOCS/Updating#Updating-%24rename">an operation</a> for renaming fields). It would be great if the driver had a <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">[BsonAlias("OldValue")]</span> attribute to avoid migration scripts (maybe I'll make a pull request).<br />
<br />
Something I liked was that I could use <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">object</span> for the type of the <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">_id</span> property instead of <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">BsonObjectId</span>. This will keep the models less coupled to the Mongo driver API. Also, the driver already has a bi-directional alias for <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">_id</span> as <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Id</span>. I don't know any C# developers who wouldn't squirm at creating a public property named _<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">id</span>.<br />
<br />
This brings me to my biggest issue with the C# mongo driver. All properties must be public. This breaks the encapsulation and SRP principles. For instance, most of the time I have no reason to expose my <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Id</span> (or <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">_id</span>) property as public. NHibernate solves this by hydrating protected fields. I would like this to be solved very soon (but there are some issues with this since there isn't any mappings).<br />
<br />
Last, it has poor support for C# 4.0 types. Tuple doesn't fail, but it's serialized as an empty object ({ }). There is also zero support AFAIK for dynamic.<br />
<br />
In conclusion, there's some room for improvement with Mongo's integration with .NET but overall I have to say I'm impressed. Supposedly Linq support is due out very soon, which will make it unstoppable (imo). Also, we haven't started using this in a full production environment yet, so there will most likely be more posts coming on this topic.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-14256075028496959252012-01-02T18:55:00.000-08:002012-01-02T18:55:05.339-08:00BDD ideas for structuring testsLately <a href="http://timkellogg.blogspot.com/2011/12/behavior-driven-development-in-c.html">I've been thinking a lot</a> about the best way to do BDD in C#. So when I saw Phil Haack's post about <a href="http://haacked.com/archive/2012/01/02/structuring-unit-tests.aspx">structuring unit tests</a>, I think I had a joyful thought. Earlier I had been thinking in terms of using my <a href="http://www.blogger.com/"><span id="goog_2068416254"></span>Behavioral NUnit<span id="goog_2068416255"></span></a> experimental project to hash out Haack's structuring idea with better BDD integration.<br />
<br />
In short, his idea is to use nested classes. There is the normal one-to-one class-to-test-class mapping, but each method under test gets it's own inner class. To use his example:<br />
<br />
<script src="https://gist.github.com/139e0c2fd267001623f1.js?file=haacked.cs">
</script>
<br />
In this example the Titleify and Knightify methods (imo two terrible uses of the -ify suffix) have corresponding test classes dedicating to testing only one method. Each method in the class (or Fact, in the case of xUnit. I actually haven't used xUnit but it seems to encourage a somewhat BDD readability) test one aspect of the method, much like the <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">it</span> method is used in rspec.<br />
<br />
I generally like Haack's test structure. For example, he points out how it plays nicely with Visual Studio's natural class/method navigation which makes the tests even more navigable. The only issue I have with it is that I dislike having 1000+ SLOC classes - tests or regular. If I were to adopt this method, I would probably break each of those inner classes into separate files (and use partial classes to break up the top class).<br />
<br />
My practice for a long time was to have one whole namespace per class under test. Consider <a href="https://github.com/tkellogg/objectflow/tree/master/objectflow.stateful.tests.unit">my tests for objectflow</a>. I actually picked up this practice from Garfield Moore, objectflow's original developer. Each class (or significant concept) has a namespace (e.g. objectflow.stateful.tests.unit.PossibleTransitions or PossibleTransitionTests). Each class in that namespace is names according to essentially what the Setup does. Some examples: WhenGivenOnlyBranches, WhenGivenOnlyYields, etc.<br />
<br />
I like the way these tests read. It's very easy to find a particular test or to read up on how a particular method is supposed to operate. But in practice this has led to very deep hierarchies, often with single class namespaces. Further, I find that creating a whole new class for each setup tends to create too much extra code. As a result, I have a hard time sticking closely to this practice.<br />
<br />
More recently I've felt a little overwhelmed with my original practice so I've evolved it slightly. Now I've started doing the one-to-one class to test mapping like commonly practiced. But each test has it's own method that does setup. For instance<br />
<br />
<script src="https://gist.github.com/139e0c2fd267001623f1.js?file=my-new-bdd.cs">
</script>
<br />
I also sometimes use this small variation of that structure where I keep the BDD sentence-style naming scheme but use TestCase attributes to quickly cover edge cases.<br />
<br />
<script src="https://gist.github.com/139e0c2fd267001623f1.js?file=my-new-bdd-2.cs">
</script>
<br />
I often use some hybrid of the last two approaches, especially if I would be using a TestCase attribute that breaks the BDD readability, I'll break the setup code into one of those Given_* support setup methods and reuse it between two different test methods.<br />
<br />
I generally like my most recent ways of structuring tests because of it's readability and ability to gain excellent edge case coverage by adding additional test cases. But I do really like Haack's structuring, so I may find myself adopting part of his suggestion and further evolving my tests.<br />
<br />
As far as this applies to Behavioral NUnit, I want to explore the possibility of a <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Describe</span> attribute that mimics the usage of rspec's <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">describe</span> method. One idea is to make the new attribute generate another hierarchical level of test casesAnonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-49136267815787782022011-12-30T18:14:00.000-08:002011-12-30T18:14:53.552-08:00Can Bad Code Ruin Your Career?I started writing this post over a year ago. I was working at a large company where I was stuck in a mouse wheel - always running to keep up but never getting anywhere. The code I had to work with was downright terrible. This, among other things, prodded me into looking for another job. While I was starting my job search I was pondering this post and decided to not finish it because I wasn't sure if some prospective employer would hold it against me.<br />
<br />
<h2>
With that said...</h2>
<br />
I just finished reading through a messy Java file. It was the usual mess of a class with a 500 line god-method (similar to the god-object) and hundreds of counts of copy and pasted code. Besides the redundant code and lack of structure the coder also used nested loops through ArrayLists when they could have used a HashSet and didn't once use generic collections, using the un-type checked versions instead. After several hours of refactoring and renaming variables I finally got to a point where I could begin fixing the bug I was after. There were absolutely no unit tests - all this code was written inline with HTML in a JSP.<br />
<br />
I spend so much time reading bad code that sometimes I wonder if I am beginning to specialize in hacks. Is it possible to read so much bad code that you forget what good code looks like? Humans are an especially adaptive species, and I think it's definitely possible that a great programmer can be forced to work in the muck so long that they forget what good code looks like.<br />
<br />
I've seen several situations where good developers produced bad code. These situations are almost always a product of an environment where features are more important than bug fixes. These companies typically invest heavily in sales and neglect IT and development costs. Or sometimes the problem is just that product management knows nothing of software development.<br />
<br />
<h2>
The 5 stages of grief</h2>
<br />
A recent coworker likened our job of working with brittle, badly designed code to the 5 stages of grief. While we were uneasily laughing about it I silently decided that this was more realistic than I wanted to believe.<br />
<br />
For instance, imagine starting a new job. In the interview process you were interviewed by intelligent, enthusiastic developers and were led to believe you were going to be working on cutting edge technologies - a dream right? When you actually get to the job you find out that the code is so backwardly complicated that its nearly impossible to touch anything without bringing the proverbial house of cards crashing down.<br />
<br />
<b>Grief Stage 1: Denial and Isolation</b><br />
<br />
Obviously the code isn't the problem, you just weren't careful enough. They probably have specific guidelines and strategies that help them be more productive. It's probably just something wrong with me...<br />
<br />
<b>Grief Stage 2: Anger</b><br />
<br />
Dammit! Who the hell even thinks of this crap? [more cursing...] Is this a god-object?? [hair gets thinner...]<br />
<br />
<b>Grief Stage 3: Bargaining</b><br />
<br />
This is typically when you start plotting potential strategies to hide the ugliness of the code. Creativity and hopeful thoughts abound. Many IT managers will talk like they are very supportive of you at this stage.<br />
<br />
<b>Grief Stage 4: Depression</b><br />
<br />
This is where the reality strikes that this stage is bad for the business plan because it involves spending less time on revenue-producing features. The IT managers that seemed so supportive now flip flop to the CEO's side and deny you the ability to cope with your problems<br />
<br />
<b>Grief Stage 5: Acceptance</b><br />
<br />
There are only two outcomes of this stage. Either (1) you accept that you can never fix the code so you decide to move on to another job or (2) you accept that you can never fix the code so you give up on trying. This is what separates good coders from bad.<br />
<br />
<h2>
Conclusion</h2>
<br />
Again, I started this post over a year ago. I've seen a lot of bad code. At my most recent job I almost took the "give up on trying" path in the acceptance stage. Luckily we hired a great older developer who snapped me out of it. I just started my new job today, I think I will be much happier.<br />
<br />
So can bad code ruin your career? My answer is a resounding YES! But it doesn't have to. Honestly, stage 5 can have better endings, but that inevitably requires understanding on behalf of management - a scarce resource.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com1tag:blogger.com,1999:blog-6849760623609771363.post-73076605600835282302011-12-28T09:51:00.000-08:002011-12-28T10:02:26.091-08:00Behavior Driven Development in C#I've been a fan of Test Driven Development since I worked in an XP shop. But every time the work starts getting bigger and more complex I always struggle to not get lost in the magnitudes of tests. I remember many early-on conversations with my <i>elders</i> about unit test naming conventions. The [method]_[input]_[output] convention starts to break down badly when your inputs become things like mocks, or if there ends up being more than 1 or 2 inputs; same with outputs.<br />
<br />
When <a href="http://twitter.com/#!/mjezzi">a coworker</a> introduced me to BDD earlier this year, it really clicked and flowed naturally. The idea of writing tests so they read like sentences out of a book or spec seems like the answer to all my questions. The ruby <a href="http://rspec.info/documentation/">rspec</a> is beautiful:<br />
<br />
<script src="https://gist.github.com/1528845.js?file=simple_rspec.rb">
</script>
<br />
The organization of the tests forces you to focus on the expectations of your test and highlight descriptive assertions. This is especially useful for complicated setups with lots of mocks, etc. I put as much of my setup code in one of those <span style="font-family: 'Courier New', Courier, monospace;">before :each</span> blocks, so that way the assertions are limited to simple inputs and one or two observations about the outputs.<br />
<br />
There's been a number of <a href="http://persistall.com/archive/2007/11/05/further-thoughts-on-bdd-in-c.aspx">people</a> in the .NET community that have attempted BDD but [imo] failed to grasp the simplicity. NBehave is a complete overhaul of unit testing that uses attributes like <i>x</i>Unit. As a result, NBehave doesn't really look at all like rspec - which really isn't a bad thing, necessarily. However, the thing I like about rspec is it's ability to describe things of arbitrary depth, which is handy when testing complex code:<br />
<br />
<script src="https://gist.github.com/1528845.js?file=complex_rspec.rb">
</script>
<br />
This spec is able to describe possible modes that the object under test can be in (complex inputs). This is made possible by rspec's arbitrary nesting depth. This is definitely a language feature that is much harder to implement in C#.<br />
<br />
My current approach to BDD in C# usually looks like<br />
<br />
<script src="https://gist.github.com/1528845.js?file=BDD.cs">
</script>
<br />
I think this is the simplest BDD layer I can slap on top of NUnit. And simple is important to me because (a) I do a lot of open source projects and I want to keep the barrier to entry for contributions low and (b) the people I work with tend to resist change. When people are resistant to change, it's hard to rationalize using something other than NUnit or introducing lots of nested lambdas.<br />
<br />
NUnit remains the most popular unit testing framework and has excellent support with a GUI runner, console runner, and IDE integration with R#, TestDriven.NET, and others. Given all that support, I would really rather not abandon NUnit if possible.<br />
<br />
<a href="http://fluentassertions.codeplex.com/">FluentAssertions</a> is a nice simple BDD layer on top of NUnit (or whatever you use). It doesn't change the structure of our spec above, but it does change the structure of our assertion to<br />
<br />
<script src="https://gist.github.com/1528845.js?file=FluentAssertion_BDD.cs">
</script>
<br />
This assertion is [imo] very clean and succinct. I like how it reads even clearer than NUnit's fluent syntax. Last weekend I was thinking about this and I decided to explore an idea to make a BDD extension to NUnit that is even clearer than FluentAssertions. The project, BehavioralNUnit for now, is hosted at <a href="https://github.com/tkellogg/BehavioralNUnit">github</a>. The earliest goal for the project was simply to use operator overloading to make the assertions even more like rspec. For instance, I want to be make the previous assertion:<br />
<br />
<script src="https://gist.github.com/1528845.js?file=BehavioralNUnit_simple.cs">
</script>
<br />
I was able to do this, but I realized that the C# compiler was insisting that this expression needed to be assigned to something, so I [haven't yet] added another concept somewhat analogous to "it" in rspec:<br />
<br />
<script src="https://gist.github.com/1528845.js?file=BehavioralNUnit_complex.cs">
</script>
<br />
This is most similar to <a href="http://nspec.org/">NSpec's approach</a> by using an indexer instead of a method. This appeals to me because I sometimes find matching parentheses to be a pain (I guess I just like ruby & coffeescript). Then again, I don't like NSpec because it feels like it was written by one of those whining .NET developers that wishes dearly he could get a RoR job - it doesn't abide to .NET conventions at all.<br />
<br />
I still have a ton of ideas to hash out with Behavioral NUnit. I'm convinced that BDD in C# can be simpler and more beautiful than it currently is. If you have input or ideas, please <a href="https://github.com/tkellogg/BehavioralNUnit">fork the repository</a> & try out your ideas (pull requests are welcome).<br />
<div>
<br /></div>Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com2tag:blogger.com,1999:blog-6849760623609771363.post-82935453348505939322011-12-26T10:30:00.000-08:002011-12-26T10:31:13.455-08:00Why I hate generated codeIf you've worked with me for any amount of time you'll soon figure out that I often profess that <i>"I hate generated code"</i>. This position comes from years of experience with badly generated code. Let me explain.<br />
<br />
<h2>
The baby comes with a lot of bathwater</h2>
In the past year I had an experience with a generated data layer where <a href="http://www.codesmithtools.com/">CodeSmith</a> was used to generate a table, 5 stored procedures, an entity class, a data source class, and a factory class for each entity that was generated. My task was to convert this code into <a href="http://nhforge.org/">NHibernate</a> mappings.<br />
<br />
The interesting thing about this work is how little of the generated code was actually being used. I'm sure, in the beginning, the developer's thoughts were along the lines<i> "oh look at all this code I don't have to write manually :D"</i>. However, after some time, subsequent developer's thoughts were along the lines of <i>"with all this dead code, it's hard to find real problems"</i>. It's funny how some exciting breakthroughs turn into headaches down the road. The table is always used, but some entities are created & read but never modified, others are only created during migrations and only read from during run time.<br />
<br />
Code generators often produce code you don't need. Since all code requires maintenance, dead code is just a liability because it doesn't provide any benefit. I always delete dead code and commented out code (it'll live on in version control, no need to release it into production).<br />
<br />
There are several professional developer communities that generate code as a way of life. <a href="http://guides.rubyonrails.org/command_line.html">Ruby on Rails</a> comes prepackaged with scripts to generate models, views, and controllers in a single command. <a href="http://www.asp.net/mvc/tutorials/older-versions/controllers-and-routing/creating-a-controller-cs">ASP.NET MVC</a> will generate controllers and views with a couple clicks. And if you've ever used either of these frameworks, you'll probably find yourself deleting a lot of generated code.<br />
<br />
<h2>
The problem of transient code generation</h2>
The issue that I keep running into with my policy of hating code generation is that it's nearly impossible to be a professional software engineer and not generate code. The most fundamental problem is compilers. When you run a compiler over your source code, it <i>generates</i> some sort of machine readable code that is optimized for various goals like speed or debugging or different platform targets.<br />
<br />
While I hate code generators, it's hard to argue how I could possibly hate compilers. They allow me to write code once and compile it several different ways and achieve different goals. Therefore, I have to introduce my first caveat - I don't hate all generated code, <i>I only hate generated source code</i>.<br />
<br />
This problem of hating generated code is complicated further by the fact that NHibernate generates source code too. You don't ever check in the code that NHibernate generates because it's done at run time. The most obvious way NHibernate generates code is the SQL that is written in the background to query & perform DML operations. (For those questioning if SQL is source code, consider how SQL is compiled into an execution plan prior to execution). It's also hard to argue that I hate this kind of code generation because it doesn't suffer from the same problems of the CodeSmith generated code. It only generates code <i>just-in-time</i> meaning that it's only generated when needed, so there isn't any extra code generated.<br />
<br />
Since NHibernate and compilers do code generation in a way that I like, I'm going to refine my statement to <i>"I hate generated persistent code"</i>. This generally means, I still hate generated code when the resulting code sticks around long enough for a fellow developer to have to deal with it.<br />
<br />
<h2>
The thin line between good and bad code generation</h2>
When is generated code persistent and when is it transient? We already decided that code generation isn't so bad when it happens during of after the compilation process. But my statement is that I hate persistent code. There are other cases of code generators generating transient source code. One such example is in <a href="https://github.com/iSynaptic/iSynaptic.Commons">iSynaptic.Commons</a>.<br />
<br />
Since C# doesn't yet (and probably won't ever) include <a href="http://insanecoding.blogspot.com/2010/03/c-201x-variadic-templates.html">variadic templates</a> or variadic generic types, writers of .NET API's often write some really redundant code to account for all combinations of generic methods or types. I know I've done it. This example uses <a href="https://github.com/iSynaptic/iSynaptic.Commons/blob/master/Application/iSynaptic.Commons/FuncExtensions.tt">a T4 template</a> to produce a C# file with a <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">*.generated.cs</span> extension. The T4 template is <a href="https://github.com/iSynaptic/iSynaptic.Commons/blob/master/Application/iSynaptic.Commons/iSynaptic.Commons.csproj#L195">executed on build</a> but not ignored from version control.<br />
<br />
I do like this approach because it takes a DRY approach to a redundant problem without much complication. Another thing I really like about this approach is that <a href="http://msdn.microsoft.com/en-us/library/bb126445.aspx">T4 templates</a> are a standard part of Visual Studio and are <a href="http://tirania.org/blog/archive/2009/Mar-10.html">executable from Mono</a> as well. As such, they can be considered a free tool that is openly available (important for open source projects) and, more importantly, are executed as part of the build process.<br />
<br />
Another thing I like about this approach is the usage of partial classes to separate the generated portion of the class from the non-generated portion. This minimizes the amount of code that is sheltered from refactoring tools (code inside the *.tt file).<br />
<br />
The thing I hate about this particular iSynaptic.Commons example is that the generated file is included in version control. I think, perhaps, this is reduced to a small pet peeve of mine since the generated code isn't wasteful and is updated on every build. Still, I would like a mechanism to (a) have the file ignored from the IDE's perspective and (b) ignored from version control. I wouldn't want anyone to mistakenly edit the file when they should be editing the T4 template.<br />
<br />
<h2>
Summary</h2>
The end result of my thought is <i>"I hate source code that is generated prior to the build process"</i>. I want to further say that I also hate generated code that is checked into version control, but this is a bit of a lesser point. However, code generation can be a useful tool; as seen in the cases of NHibernate and T4 templates. But even still, code generation should be used wisely and with care. Generating excess code can become a liability that detracts from the overall value of a product.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-81851478173928340662011-12-01T20:59:00.001-08:002011-12-01T21:24:03.964-08:00Defining WatergileAt the place of my current employment we've had a layer of management placed above us that fervently preaches the mightiness of agile. This management devotes much lecture time into informing us the proper procedure of planning a product. First you gather requirements and architect the entire system and write detailed requirements documents - good enough that developers don't need to refine them any further and QA knows exactly what to test. When requirements are written for the entire system - 12-24 months in advance - then you begin coding. After you're done coding, QA begins to test.<br />
<br />
To be clear, anyone reading the previous paragraph should be scratching their head and thinking to themself, "gee, that sounds a lot like waterfall". Well it is, hence the portmanteau <i>watergile </i>(we considered agilfall but it just doesn't roll off the tongue as well).<br />
<br />
The trouble is, even though we coined the term just recently, this watergile thing is a frigging pandemic. Every time I crack open a fresh copy of <a href="http://www.sdtimes.com/">SD Times</a> there seems to be some guy telling you that you need to be measuring KSLOC and a billion other software metrics but at the same time claiming that agile is the only way. It wouldn't be so scary except that this is the source of direction for software development managers.<br />
<br />
It's no wonder <i>watergile</i> is so widespread, IT managers are fed a constant stream of B.S. mixed messages. How could anyone make sense of any of it without dismissing most of it? The truth is, waterfall is hard and so is agile. Anything in between is just ad-hoc and setup to fail. If you are a development manager and reading this, find those tech magazines on the corner of your desk and show them to the recycling bin. They're worthless and distracting to progress.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-55842059131772632562011-11-06T21:35:00.000-08:002011-11-06T21:35:27.701-08:00The Pain and Glory of CI don't normally write much C code, but this past week I was fiddling around with it this past week to solve some programming puzzles. When I say C I mean straight C (without the ++ or #). Completely un-object-oriented; just structures, helper functions and malloc/free. It took me 3 days (a total of probably 9 hours) to write a fully functional 250-300 SLOC solution to a puzzle (complete with huge memory leaks). This all brings me to the burning question - who would ever want to write programs in C?<br />
<br />
C++ has developed over the years. I recently looked at some of the enhancements in C++11 which include the auto keyword (like var in C#), better reference counting "smart pointers", lambdas and closures. Obviously, C++ is developing and progressing. C hasn't had a spec change since 1999, and even then it wasn't exactly dramatic. We still don't have any OO or reference counting pointers.<br />
<br />
Have you ever tried interfacing with a library in C? It's very cumbersome. You have to read all the documentation and call the right my_library_object_*() functions at the right times. Everything is hands-on, nothing is left to imagination. You have to remember what memory you allocated so you can free it sometime later when you're sure you don't need it anymore (and then recursively free sub-structures and arrays).<br />
<br />
I think anyone can see warts in C. But its easy to forget the simplistic beauty. I mean, there aren't many operators in C, and there's only one way to cast. I mean, sure, you still can't create & initialize a counter variable inline in a for-loop. But the complex syntax of C++ is scary in comparison with all it's member::accessors, template<t classes="">, 5-6 ways to cast a variable and a slew of gotchas. Sure, C has it's share of gotchas, but the language is so small that anyone who's spent any significant time programming C can list most of them out for you (probably not so true with C++).</t><br />
<br />
So why not C#? Well, it's freaking slow!! Think about when people were converting their business apps from VB6 to C#. Sure the maintainability of the code improved by leaps and bounds, but almost everyone noticed the performance difference and wondered how the same program could be so slow.<br />
<br />
Recently Microsoft unveiled some information to developers about the upcoming Windows 8 release and it's metro interface. One of the biggest surprises to developers is how hard Microsoft is trying to sell C/C++ and how C#/.NET is falling by the wayside. The driving factor is that Apple has snappy user interfaces and Windows Forms are known for being slow and boring. So Microsoft created a new WinRT UI toolkit for Windows 8 that intends to never block the UI thread. Operations that take longer than ~50ms should use Async code so that the UI can continue to feel responsive. (This sounds eerily similar to <a href="http://nodejs.org/">Node.JS</a> but with a lot more code).<br />
<br />
Obviously Microsoft wants developers to develop faster apps by going back to C/C++, maybe we should consider taking them seriously. But I think the more likely direction is development being done primarily in one of the common dynamic languages like Ruby/Python/Node.JS with certain code that needs speedup written as C modules. All of those general purpose scripting languages are written in C (not C++) and interface very well with C. I've seen lots of math-intensive Python libraries being composed partly of C code (some with increasing portions written in C). I could also see the popularity of Node.JS increase if it was applied to more than web/networking apps but also non-blocking UI. (After all, this is basically what WinRT is).<br />
<br />
I don't know about you, but I'm going to be spending some time tuning up my C/C++ skills. History has been known to repeat, and I think it is now repeating yet again.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-35039978542593500372011-10-31T21:33:00.000-07:002011-10-31T21:33:48.456-07:00Occupy Wall Street Is Not StupidEarlier today I was talking with someone today who exclaimed, "Occupy Wall Street, that's so stupid!". I then proceeded to explain to them that OWS is trying to say <i>"hey, this capitalism thing isn't really working right now". </i>It's not to say that capitalism never worked, it's just pointing out that there are some significant holes in it right now.<br />
<br />
I believe that by now, most people (except some in Boulder) realize that communism has also failed. Now, communism didn't fail because <i>God hates communists</i>. It failed because it wasn't maximizing the total economic prosperity of all people. The people behind OWS have also realized [, I naively assume,] that capitalism in America is also no longer maximizing the total economic prosperity.<br />
<br />
In America today you see thousands of families that incurred large amounts of debt to a disgustingly rich minority. This rich minority (an oligarchy) forced these families out of their homes and into slavery. You might recognize that this looks a lot like the economic system that capitalism replaced - feudalism.<br />
<br />
OWS protesters are also crying out about the death grip that rich and powerful businesses have on our federal government. Some even claim that presidential elections are completely rigged (I probably wouldn't go that far). Either way, the government that our American forefathers created is completely absent and void from our current government. We've become so obsessed with being the most powerful country that we sacrificed the values and virtues that made us who we are.<br />
<br />
The Occupy Wall Street movement is right, our system is broken. Yes, there are many broken systems out there, but that's not a reason to not change them. Protest is an important political mechanism that has been proven to work in the past. We need it to work now. The only problem I have with OWS is that it seems to be an incohesive jumble of complaints with no real answers. But I suppose that's where real change begins.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com1tag:blogger.com,1999:blog-6849760623609771363.post-87054106998109576312011-09-30T19:01:00.000-07:002011-09-30T19:01:51.183-07:00Quiet TimeRecently, we instituted a "core hours" policy among our developers that essentially equates to 4 hours of quiet time every day. During the hours of 10-12 and 2-4 developers aren't allowed to interrupt each other, nor can QA, product managers, or anyone else in the office interrupt developers. If you need help on a problem you have to either work through it on your own or wait until after the <i>quiet time</i>.<br />
<br />
The policy hasn't been in effect very long, but I've immediately noticed a significant jump in productivity. I would say I'm 1.5-2 times as productive now that I'm not getting interrupted every 15 minutes. I've also notice that I just plain enjoy coming to work more now.<br />
<br />
When we were talking about instituting the policy some were worried that it would be a problem that you couldn't clear up issues and roadblocks immediately. In practice, however, I think it isn't too much to ask everyone to wait [up to] two hours to clear roadblocks. In fact, it ends up forcing developers to solve their own problems.<br />
<br />
When I first started with this company I was isolated in a room by myself with entire days to myself. The isolation was too much; I often felt like I was being confined in a prison. Obviously I'm not advocating that total isolation is any kind of real solution. It's impractical to suggest that developers can complete their work successfully in total isolation. It takes a lot of dialog to produce quality software. But it's also impractical to suggest that they can get any work done when they're being pestered every 5-30 minutes.<br />
<br />
I highly recommend some sort of <i>quiet time</i> in any work place. In my opinion, the benefits are definitely not limited to just software engineering either.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-22239829781054865842011-09-15T21:19:00.000-07:002011-09-15T21:27:04.083-07:00AutoMapper And IncompletenessThis is part 2 of a series. Read <a href="http://timkellogg.blogspot.com/2011/09/view-models-automapper-and-law-of.html">part 1</a><br />
<br />
Earlier I talked about the Law of Demeter and how view models help us better adhere to the Law of Demeter. I also briefly outlined how AutoMapper makes view models practical. While AutoMapper is a great tool, it isn't completely fulfilling. Let me explain<br />
<br />
As I pointed out previously, some of the behaviors in AutoMapper make it feel incomplete. The first is that you can't map two view models to the same model and back.<br />
<br />
A much bigger problem with AutoMapper is that view models can't extend models. I'm not sure why they decided to disallow this usage, but it causes a cascade of code duplication (very un-DRY). Take a look at these classes:<br />
<br />
<script src="https://gist.github.com/1221098.js?file=ModelsAndViewModels.cs">
</script><br />
<br />
There are a few things wrong here. Age is a nullable int on the model but the view model has just an int. If a null slips through this could cause a crashing error. While AutoMapper has an AssertConfigurationIsValid method, it doesn't test for this sort of case. You'll have to make unit tests for this, luckily you can use <a href="https://github.com/tkellogg/NetLint">NetLint</a> to easily test for these sorts of flukes.<br />
<br />
Another issue is the validation attributes. The facts that account codes look like CO11582 and that all accounts must have a name are descriptors of the domain (which the model is modelling). They aren't facts about the view (although they have to be expressed in the view), they are part of the model. Every time you create another AccountViewModelX derivative AutoMapper requires you to copy these attributes. This is a massive failure in the attempt to keep code DRY.<br />
<br />
Another issue I have is when I'm creating a view model I'm not sure what properties need to be created. I usually have to split the window and copy properties from model to view model (this screams obscenities at the idea of DRY code).<br />
<br />
One solution that I keep coming back to is to have view models extend models. For instance, see this implementation:<br />
<br />
<script src="https://gist.github.com/1221166.js?file=gistfile1.cs">
</script><br />
<br />
Here, you don't have to type out all those properties a second (or third) time. They're just available. You also won't make the mistake of marking Age as non-nullable or forget to copy the validation attributes. It's all done for you by the compiler - no need to write extra tests.<br />
<br />
There are still some issues with this approach, and other approaches (such as encapsulation) that you can take. Perhaps there will be a part 3.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-23680665145451248712011-09-12T20:14:00.000-07:002011-09-12T20:14:37.516-07:00View Models, AutoMapper, and The Law of Demeter<div>
The <a href="http://haacked.com/archive/2009/07/14/law-of-demeter-dot-counting.aspx">Law of Demeter</a> was created for the intent of simplifying object hierarchies and structures. Obviously it's not a blanket sort of law (doesn't seem to apply to <a href="http://www.themomorohoax.com/2009/02/25/how-to-write-a-clean-ruby-dsl-part-2-line-by-line-with-machinist-rails">DSL's </a>or fluent interfaces). But it is handy to keep in mind when modelling a domain. </div>
<div>
<br /></div>
<div>
A classic example of a shortcomings of the Law of Demeter is name example: passing a model to a view that has a name object (<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Model.Name.First</span>, <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Model.Name.Last</span>, etc) versus passing a flattened view model (<span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Model.FirstName</span>, <span class="Apple-style-span" style="font-family: 'Courier New', Courier, monospace;">Model.LastName</span>, etc). I think this is a great application of view models.</div>
<div>
<br /></div>
<div>
I like the idea of view models because they're a great way to express view-specific business logic. The FirstName/LastName is an example, but they're also great for holding data necessary to populate drop down lists and summary views. Beyond code, view models are also a good example of the .NET community's ability to innovate new solutions to old problems (akin to <a href="http://timkellogg.blogspot.com/2011/08/parenthetical-thesis-on-rubynet-or.html">my thoughts about the ruby community</a>) </div>
<div>
<br /></div>
<div>
<span class="Apple-style-span" style="font-size: large;"><b>Yes, But...</b></span></div>
<div>
<span class="Apple-style-span" style="font-size: large;"><b><br /></b></span></div>
<div>
While I definitely understand the benefits of view models, I'm still trying to figure out the best way to use them. When first creating view models the urge is to write and populate them by hand. This quickly becomes very tiresome. Enter <a href="http://automapper.codeplex.com/">AutoMapper</a>. </div>
<div>
<br /></div>
<div>
AutoMapper is an object-to-object mapper designed very specifically for flattening models into view models. It bases it's decisions on conventions and provides a fluent interface for the remaining anomalies. It is a savior for those writing view models by hand.</div>
<div>
<br /></div>
<div>
AutoMapper works only in one direction. You take an existing model and map and migrate the data into a view model. Going backwards; however, is another story. One big limitation of AutoMapper is that you can't map from two different source types to the same destination type. This makes it difficult or impossible to use AutoMapper to do bidirectional mappings (for instance, if you want to use AutoMapper when updating the model from FormCollection).</div>
<div>
<br /></div>
<div>
There is quite a bit more I want to say on this matter, which I will continue in a second part</div>
Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-58506742610128673882011-09-05T18:48:00.000-07:002011-09-05T18:48:40.778-07:00Introducing comboEditableI'll admit, <a href="https://github.com/tkellogg/comboEditable">comboEditable </a>is an extremely dry name for an open source project (I would have used something like <i>Project Bierstadt</i> but it's not really that descriptive). Like everything else I develop and share publicly, this came out of necessity.<br />
<br />
In Windows there is a UI concept of an editable combo box. Basically you're given a drop down list of options and if you can't find the option you're looking for, you just type in another (see the <a href="http://tkellogg.github.com/comboEditable/">demo</a> if you're having trouble visualizing). This concept <i>does not exist</i> on the web or anywhere outside Windows applications. I assume that UX designers across the globe unanimously decided that an editable combo box is a UI kludge, but I still think it's a handy control.<br />
<br />
It is an unintrusive jQuery plugin that uses the regular HTML DOM as input and transforms into an editable combo box (a text box, hidden field and several divs, if you're wondering). The <i>unintrusive</i> part means that if scripts are disabled, the user still gets a combo box, just not an editable combo box.<br />
<br />
If you find yourself in need of an editable combo box, head over to the <a href="http://plugins.jquery.com/project/comboEditable">jQuery plugin page</a> or download it at <a href="https://github.com/tkellogg/comboEditable">github</a>. Also, take a look at the <a href="http://tkellogg.github.com/comboEditable/">demo</a> to see usage.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-91998143005635334582011-08-29T21:24:00.000-07:002011-08-29T21:24:29.476-07:00Parenthetical Thesis on Ruby.NET (or IronGem (or whatever the kids call it these days))Since college I've always been a huge fan of dynamic languages. I was really into Python for a long time and in the past year or so I've picked up Ruby. It's well known that the open source/dynamic language world has always looked down on the .NET/Java world as some sort of inferior. While having a conversation with a colleague about ruby versus .NET I stumbled on a conclusion.<br />
<br />
Ruby has some great features like mixins, monkey patching, <i>a <a href="http://en.wikipedia.org/wiki/Read-eval-print_loop">REPL</a>. </i>I also love how blocks make closures such an accessible and natural way to program. Ruby makes easy things easy and hard things fun.<br />
<br />
On the other hand, C# is one of the most beautiful typesafe languages (although F# is gaining favor with me). Linq and expression trees provide functionality that you literally cannot reproduce in dynamic languages (it requires knowledge of types, which dynamic languages theoretically shouldn't care about). With the crazy stuff that people are doing with expression trees (building SQL statements, mapping objects, selecting properties, etc) it makes it hard to say I'd rather be doing ruby.<br />
<br />
While C# has some analogous ruby constructs (extension methods are kind of like a lesser form of monkey patching), it still suffers from some of the classical faults of static languages (there can be a lot of extra code just to deal with types and to play nicely with the compiler). At the same time, the compiler also writes tests for you (a contract states you will have these methods, yet in ruby you can't ever be completely sure they'll actually be there. Something that you'd have to write unit tests for in ruby).<br />
<br />
The conclusion I came to was that, at this point in time, there really isn't a compelling reason why ruby is better than .NET or vice versa. Except for one thing - the communities. The ruby community is nearly too much fun. In Boulder, where I live, there are several companies that host regular hackfests. There are also annual ruby conventions where people get together, socialize, and share new ideas. In the .NET world we have some of those perks, but we're notoriously laiden with deadbeats. I can't tell you how many lame coworkers I've worked with that have little interest in improving themselves or the code they write. While in the Ruby world, they're not just interested in themselves or the code they write, but also <a href="http://codeforamerica.org/">in the community around them</a>.<br />
<br />
Despite all the debate, I'll probably keep my current job. I love the people I work with and I like participating in the .NET open source world (there really aren't any deadbeats in any sector of the open source world, by definition).Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-5872980606161901902011-08-27T22:25:00.000-07:002011-08-27T22:25:54.810-07:00Launching personal websiteI spent some time today and solidified my personal website (<a href="http://tkellogg.github.com/">http://tkellogg.github.com</a>). I'm pretty excited about this website just because its a great demonstration of single page apps. Each of my main links doesn't actually take you to a different page - it uses a JavaScript routing engine (<a href="http://documentcloud.github.com/backbone/">backbone</a>) to load and display new content.<br />
<br />
I do have some plans for the site, but there are so many more important things to deal with these days. But if I can get to them I want to start a picasa site and load images into the site using the gdata api (like how I load blog posts now) and also integrate with github to list out my repositories and activity.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-76395485591671650792011-08-08T20:18:00.000-07:002011-08-08T20:18:51.020-07:00Maybe Node isn't so badI know in <a href="http://timkellogg.blogspot.com/2011/06/got-backbone.html">previous</a> <a href="http://timkellogg.blogspot.com/2011/05/hipster-developers.html">posts</a> I bashed <a href="http://nodejs.org/">Node.js</a> a bit. I've done some thinking about it and I was struck by <a href="http://codeofrob.com/archive/2011/04/30/5-reasons-to-give-node-js-some-love.aspx">a revelation</a>. If you write a Node app that serves to a browser you can use the same code on client & server. That means you can use frameworks like <a href="http://documentcloud.github.com/backbone/">Backbone</a> to manage your business logic on both on the server and on the client inside a browser.<br />
<br />
The implications for this are huge. I've toyed with the idea of using Backbone + ASP.NET MVC together for a while now but I kept tripping up on all that code duplication between Backbone models and C# models. Node could be what launches the browser into a universal rich client host (and yes, HTML5 will help too).<br />
<br />
The other crazy idea I had about using node is that this means less languages to learn. Imagine if you wrote JavaScript intensive apps with Node and backed it up with <a href="http://www.couchbase.com/">couchbase</a> on the DB end. You would have JavaScript in your view, Javascript for business logic and JavaScript in the DB. The learning curve for a new developer to become productive would be the smallest learning curve that IT has seen in decades, probably for all time. This could change the landscape of IT forever. It wouldn't be such a bad idea to build a development team around that concept.<br />
<div><br />
</div>Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-19862792124182280232011-07-27T21:08:00.000-07:002011-07-27T21:08:48.253-07:00Git is a platformThis evening I stuck my head in at <a href="http://quickleft.com/">quickleft's</a> <a href="http://quickleft.com/blog/tag/hackfest">hackfest</a> downtown boulder. They gave a great intro to ruby & sinatra. Sinatra is mind-bendingly simple. It makes you wonder why you've been doing anything but sinatra.<div><br />
</div><div>Anyway, while I was playing around at the hackfest they introduced heroku, which is a cloud platform for ruby. Heroku uses git to let you manage your application's files on the server. Pushing a brand new repo creates a new domain name and sets up the infrastructure for your app. They built a very cool application on top of the <i>git platform</i>.</div><div><br />
</div><div>Github has been doing this for a while. I blogged earlier about <a href="http://timkellogg.blogspot.com/2011/02/internal-secrets-of-git.html">github</a> and the things they've done with git. The most public things include git as a blogging/wiki engine as well as a static website generator (github pages). You can also fork <a href="https://github.com/icefox/git-achievements">git-achievements</a> and broadcast your mastery over git, <a href="http://tkellogg.github.com/git-achievements/">like I did</a>. Honestly, the things you can do with git are endless since it is, after all, nothing more than a versioning filesystem in user space.</div><div><br />
</div><div>I think this is the biggest thing that separates git from other version control systems. No one has done anything with SVN beyond simple pre or post-commit hook scripts. TFS has a lot of application infrastructure built <i>around</i> it, but it doesn't build <i>on top</i> of it's version control system. Neither does mecurial or bazaar, even though they are also distributed version control systems. </div><div><br />
</div><div>The git folks really focused on defining git as a standard rather than an application. By that I'm referring to how they defined objects, trees, packfiles, etc (see <a href="http://progit.org/book/ch9-0.html">progit</a>) instead of focusing on developing an application. For much of it's lifetime git was nothing but a hodgepodge of shell scripts and C libraries. Now days there are <a href="https://github.com/igorgue/git-sharp/wiki">several</a> <a href="http://www.jgit.org/">varying</a> <a href="http://libgit2.github.com/">implementations</a> <a href="http://deadpuck.net/blag/serving-git/">of</a> <a href="http://git-scm.com/">git</a>. The fact that git is so widely programatically accessible is making it insanely easy to leverage inside programs. I'm still waiting for a .NET app to do something big with git#...or maybe I could.</div>Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-79330194853281364772011-07-10T17:05:00.000-07:002011-07-10T17:05:18.893-07:00Semantic versioningI've seen some interesting software version sequences. Like Windows 3, 3.1, 3.11, 95, 95, ME, XP, Vista, 7. Or Oracle DBMS v5, v6, 7, 8, 8i, 9i, 10g , 11g (what does the <i>g</i> mean??). I've seen all sorts of version schemes to designate major versions, minor versions, patches, and other types of releases. (The worst ones are always when marketing gets involved).<br />
<br />
<a href="https://github.com/mojombo">Tom Preston-Werner</a> formalized the major-minor-point release (<i>X.X.X</i>) scheme at <a href="http://semver.org/">semver.org</a>. I highly recommend anyone who considers themselves a professional developer to read every word in the article at <a href="http://semver.org/">semver.org</a>. The beauty of semantic versioning is that there isn't anything new or innovative about it at all. It's all what you already know to be true. All versions <1.0.0 are development versions. Once 1.0 hits, the public interface is solidified. If and only if you break backwards compatibility you have to increase the major version. Minor versions and point releases (1.X.0 and 1.0.X) are for various levels of new features and bug fixes.<br />
<br />
When you release software labeled with semantic versions you make it easy for people to quickly asses how significant the release is (I might skip a point release and upgrade to minor releases, but I might avoid a major release due to the incompatibilities it might cause). It also forces the developers to exercise restraint in breaking compatibility with previous releases.<br />
<br />
The trouble with semantic versions in the corporate world is that marketing always has ulterior motives. They want to release a major version to make the product feel alive; they want to downplay breaking changes to a minor version to keep customers; or they want to introduce new terms that mean nothing to the average user (XP for <i>eXPerience, </i>Vista because it sounds cool). Those names are great for development code-names but they detract from a buyer's experience (I use the term buyer loosely to mean any potential user) in determining compatibility between products.<br />
<br />
In .NET assemblies, there are four segments supported with the AssemblyVersion and AssemblyFileVersion attributes (major, minor, build number, revision). This seems fine until you want to release alphas, betas and release candidates. The semantic version for a 1.0 beta release would be 1.0.0beta1 indicating that this is the first beta for the 1.0.0 release (you can use any string of alphabetical characters, not just <i>beta</i>). In a .NET assembly <a href="http://stackoverflow.com/questions/64602/what-are-differences-between-assemblyversion-assemblyfileversion-and-assemblyinf">you do this as follows</a>:<br />
<br />
<pre class="brush: csharp">[assembly: AssemblyVersion("1.0.0")]
[assembly: AssemblyFileVersion("1.0.0.253")]
[assembly: AssemblyInformationalVersion("1.0.0beta1")]
</pre><br />
The new attribute here is obviously <a href="http://msdn.microsoft.com/en-us/library/system.reflection.assemblyinformationalversionattribute.aspx">AssemblyInformationalVersion</a>, which is used to specify more arbitrary strings. It will show up in the Windows properties dialog as the assembly version (otherwise <a href="http://msdn.microsoft.com/en-us/library/system.reflection.assemblyversionattribute(v=vs.71).aspx">AssemblyVersion </a>will be used). Also, the <a href="http://msdn.microsoft.com/en-us/library/system.reflection.assemblyfileversionattribute.aspx">AssemblyFileVersion </a>is used to indicate build numbers. So while working on the 1.0.0 release, we also have a continuous integration environment like <a href="http://www.jetbrains.com/teamcity/">Teamcity </a>or <a href="http://hudson-ci.org/">Hudson </a>building the code each night and incrementing the build version. However, continuous integration environments shouldn't need to have any impact on what you actually tag the version as.<br />
<br />
As Tom says in the article, kinda sorta following the standard doesn't reap much benefit. But once we all start releasing software that conforms <i>exactly</i> to this standard, then users can more efficiently understand which two components are compatible and which aren't. I believe this applies to all software, not just software that supplies a public API.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com2tag:blogger.com,1999:blog-6849760623609771363.post-27511755348521468662011-06-28T21:09:00.000-07:002011-06-28T21:09:46.795-07:00Got a backbone?Earlier, I posted about those lame <a href="http://timkellogg.blogspot.com/2011/05/hipster-developers.html#links">hipster developers</a>, as I call them. Mainly, I just find it a little hard to believe that anyone can create a truly scalable JavaScript app using <a href="http://nodejs.org/">node</a>.<br />
<br />
Recently I stumbled into <a href="http://documentcloud.github.com/backbone/">Backbone</a> (or rather I kept on hearing about it and finally checked it out). Backbone is a bare bones MVC framework for JavaScript that is meant to help give your JavaScript apps structure without weighing them down. Also, more important, is that Backbone is by no means mutually exclusive with jQuery. Actually they compliment each other quite nicely.<br />
<br />
Back to those hipster developers. I don't often like to admit that a badly dressed 20-year-old can be right, and I still won't go so far as saying node.js is really a presentable solution for anything on the server, but the fact that they're expanding the infrastructure around JavaScript is really pushing me to think about how I can evolve my own .NET work. For me, Backbone is where it starts.<br />
<br />
<span class="Apple-style-span" style="font-size: large;">An Answer to Uncontrollably Messy JavaScript</span><br />
I've written a lot of pages with big long blocks of jQuery chains and anonymous functions. It's such a huge pain to maintain or refactor that I sometimes end up rewriting. Part of the problem is just simply that the code is messy. But even when I break it down into smaller nugget sized functions I still have a fist-full of spaghetti code that is prone to unchecked regressions. I definitely need to test my code but<br />
<br />
Backbone lets you organize your code into Models, Views and Controllers and Collections. If you go all the way with Backbone, you're going to be creating pageless apps where you load the page the first time, and you never reload the page (like GMail). Everything is data fed to the page via JSON services. Controllers let you bind bookmarks to functions (i.e. when a link gets clicked where href="#!/inbox" the link gets routed to an inbox function and handled there). Views bind models to HTML. They also keep the models bound to the HTML, so when newer fresher data arrives, the models are rebound to the page where necessary.<br />
<br />
By modularizing code according to the MVC pattern, unit testing becomes significantly easier. Most of your normal issues like mocking the DOM & XHR become less important because your code is broken into smaller pieces. Besides being easier to test, it's just plain easier to understand also.<br />
<br />
When testing, if you do require mocking facilities, I've heard that <a href="http://sinonjs.org/">SinonJS</a> is excellent for all types of mocking, and comes with built in server & XHR mocks. Also, a coworker is pushing me towards <a href="http://behaviour-driven.org/">Behavior Driven Development</a> and so <a href="http://pivotal.github.com/jasmine/">Jasmine</a> is a natural winner for a test framework.<br />
<br />
I've heard people stress that Backbone is for web applications, not web sites. But at the same time, I don't think you need to go completely single-page to use Backbone either. In .NET, I don't really want to go single-page because MVC provides so much. But some of my pages that involve several page states could be dramatically simplified with an MVC approach. At bare minimum, I want to be able to simplify and test my client-side logic.Anonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com0tag:blogger.com,1999:blog-6849760623609771363.post-90779121335745550302011-06-26T16:34:00.000-07:002011-06-26T16:34:46.832-07:00Introducing NetLintLast week our QA guys wrote up a bug that one of our new pages wasn't working. After a little investigation I figured out it was just a JavaScript file that was inadvertently merged out of existence while resolving merge conflicts. We also had something like this happen where the app would run locally on developer boxes but would fail miserably when we deployed to the test environment.<br />
<br />
I don't really like giving the QA guys an excuse to blemish my reputation with bug reports, so I threw together a little tool to prevent this from ever happening again. Enter <a href="https://github.com/tkellogg/NetLint">NetLint</a>...<br />
<br />
NetLint processes Visual Studio project files (*.csproj, *.fsproj, etc) and compares files that exist in the project file and the files that actually exist on disk. So if a JavaScript file exists on disk but isn't in the project file, NetLint will throw an exception summarizing this and any other discrepancies.<br />
<br />
I also setup NetLint with simple file globbing functionality, so all files under bin/ and obj/ are ignored by default (you can also do custom patterns). I run NetLint from a unit test, so whenever anyone resolves merge conflicts they will instantaneously know if they missed a file.<br />
<br />
The future of NetLint will be a staging ground for <a href="http://devlicio.us/blogs/krzysztof_kozmic/archive/2011/03/09/testing-conventions.aspx">testing conventions</a>. I'm licensing it under the MIT license, so hopefully no one should have any reservations due to licensing. I also created <a href="http://nuget.org/List/Packages/NetLint">a NuGet package</a> to make it even easier to useAnonymoushttp://www.blogger.com/profile/12067028473388265245noreply@blogger.com1