MacBook Pro

I've been wanting a new laptop for a little while, now. I haven't been able to find the perfect laptop, yet, but the MacBook Pro comes fairly close.

Here are my requirements:

• I need Windows for Visual Studio, SQL Server, and IIS. I guess IIS is optional, but since I have Windows already, might as well use IIS. This is mainly because I really like the way VS works, and SQL Server is fairly easy to use and configure.
• I need an SSD drive. This is fairly obvious. The speed improvements are just crazy good. As an anecdote, my wife's machine was constantly trashing with only 4GB to offer to Windows 8 and her 50 Chrome tabs. Instead of upgrading her to 16GB of RAM, I decided to upgrade to an SSD. Now it's fairly responsive even with 100 hard faults/second. On the plus side, anything going to disk is fast. 
• I want 16GB of RAM. I know I only gave my wife 4GB, but I plan on running VMs, and I want to give SQL the memory it loves. I really would like to upgrade this in the next 3 years as well.
• I want over 4 hours of battery life. I had a nice laptop that only had 2 hours of battery, and it was pretty annoying at times.
• I want a 15" or 17" screen. I already think 17" is small, but they don't make any reasonable 19" laptops.
• I want a CPU with over 2000 on the single thread PassMark benchmark. Multithreaded benchmarks no longer mean much to me. I occasionally use 2-3 cores, but I don't really care how much the 8th core runs. I want whatever I'm doing to be fast, and I can't do that many things at once.
• I want a trackpad that works well. I've used a few crappy trackpads. I had it permanently disabled on my other laptop, it was so bad.
• I want 1920x1080. I think this is a perfect fit for a 17" screen. I guess I can settle for 1600x900 on 15", but I like having 500px on each side of the screen. This pushes right up against my medium line lengths. About 5% of my lines need more space. Retina screens are no good since they make everything super tiny or blurry.

The MacBook Pro meets most of these requirements, but misses a few:

• The 15" comes only with a retina screen.
• The RAM is soldered on. I don't really want to be removing solder from a $2500 laptop.
• Doesn't run Windows out of the box.

If I put all my requirements together, the price range is around $2000-$2500. So I decided to just go with a MacBook Pro and see how it goes. Apple has a 14 return policy, so I'm going to see how it works in these two weeks.

So far, I've had these issues: 

• I'm using Parallels since VirtualBox doesn't support Retina, and makes everything super blurry.
• Battery life at 90% CPU and 90% disk was only 2 hours. That's a far stretch from the 8 hours advertised. I'm going to try using it with a lighter load today and see what happens. 
• The different keyboard shortcuts are fairly annoying. For now, I'm just going to run everything in a VM. I also installed Karabiner to switch alt and win keys for the VM.

ASP.NET vs RoR

ASP.NET? Why would anyone compare RoR to ASP.NET? The startup community has mostly dismissed ASP.NET based on the belief that Rails is a much more agile stack. Much of this applies to the Java stack as well.

One of the biggest reasons to not use ASP.NET is because it came from Microsoft. Some common gripes include not open source, letting Microsoft control your destiny, cost, and Windows servers. While this may have been true, Microsoft has recently open sourced a large portion of their system, and the cost is negligible for the first 3 years with BizSpark. Plus, the costs aren't that bad. When we look at the efficiency of .NET vs any Ruby platform, .NET is clearly fast enough to cover the Windows licensing costs. Some of the more adventurous ones will use Mono to run .NET on Linux, but I prefer the stability and speed of Microsoft's .NET over Mono. I will agree, however, Windows servers are annoying to manage. I haven't had experience with Linus machines, yet, but I don't like giving up so much of my RAM just to run Windows. I don't think Server Core is 100% ready, yet, since many non-Microsoft components still require the GUI.

Once we get over the fear of Microsoft, many people cite Ruby as one of the biggest advantages. Ruby being a dynamic language makes it the most agile. I wholeheartedly disagree with this one. I have loved Python for a few years, but that was because I was using C++. C# (and even VB.NET) has changed quite a bit over the years. C# is capable of being dynamic as well, but I don't like what I give up there. With type inference, lambda functions, extension methods, and LINQ, it's not that bad anymore.

Plus, I think the biggest gain to productivity is the compiler. I agree waiting 10 seconds to see a change reflected in the browser is annoying, but there's a huge number of tests I don't have to write because of the static type system. When I change something, I will know all the places I broke because the compiler knows. Moreover, I can't make typos anymore. Between IntelliSense and the compiler, it's impossible to type in a variable that doesn't exist unless I choose to be in the dynamic land. Those in the dynamic world would need to write a unit test for these things. I get them simply by declaring a class.

A huge draw for Rails is convention over configuration. I think this is great to a certain point. The obviously correct choices should be done by convention, or at least auto-generated configuration. However, I think there are many obscure conventions that at not obvious, and more importantly, not how I want them. For example, I don't want my view files in a separate folder from my controller files. Why can't I have them right next to each other like ASPX and code behind files? They are logically tied together, and separating them by location is just annoying. (This also applies to ASP.NET MVC, and it's very annoying in large projects.) Instead of setting a bunch of configurations to get started, I have to learn a bunch of conventions. Sure, like all other noobs, I will start by just glossing over all the magic, but they're going to come bite me later. I don't like getting bitten by decisions I didn't choose to delegate.

The huge library of gems lets developers get additional functionality for nearly free. This is also cool to a degree. The cost changes from re-inventing the wheel to learning documentation and traps. At least having gems gives me the choice of outsourcing the functionality or not. That's cool. The .NET world is sorely missing these gems. Something that helps alleviate this problem is that many integrations are now moving out of the core stack into JS libraries or web services.

But I am willing to try out Rails for a while to see if it's really as amazing as the community thinks it is.

From Drupal to Rails

That fling with Drupal didn't last too long. I had hoped to do more with Drupal, but as I got to know it more and more (I addressed a few bugs we had), it became apparent that Drupal is a really good CMS. Since we are not centered around content, it didn't make sense to have a CMS there. Let's take a look at a few things wrong with Drupal.

The way Drupal was designed, it has horrible performance, but covers over it all with a lot of caching. This was really painful for me since I had to wait 10 seconds for my changes to render since I was constantly clearing the cache. I probably didn't need to clear all of the cache, but I didn't want to spend the time to figure out which caches I needed to clear. The main problem here is that we won't be able to cache much. (Or we will have a lot of cache invalidations flying all over the place.) We are expecting our content to constantly change.

No foreign keys. I had just come from a shop that doesn't use foreign keys, but the naming convention made it obvious which columns where referring to others. All foreign keys must have the same column name as the primary key. Drupal also has this convention, but the primary key column names are all three letters long, two of which are "id". So Node table has key "nid". That's cool if I only had 26 tables being referred to, but sadly there were 3 tables that could be the target of "aid", and I had no idea.

Drupal is very modular. With the modularity comes a lot of redirection. I know the hot thing for the past decade has been IoC, DI, interfaces, and all that stuff. That makes it hard for me and my IDE to follow what's going on. Things just happen automagically, and it hurts someone new like me.

Also along the lines of modularity, Drupal makes it sort of easy to assemble different pieces together. This is cool, but that means I had a lot of extra code that I wasn't using because we're just using one facet of many pieces to make things work. Free stuff can't be bad, right? But now there's more indirection, more tables filling up my object explorer, more possible security holes, and more options to possibly screw me over next time.

These reasons make it hard for me to work with Drupal. Nothing wrong with Drupal. We were just using it for the wrong purpose. So now we're moving to a lower level framework which will be fighting with me less.

So we were going to choose a new framework. Since I'm here to write an awesome app, not reinvent plumbing, I decided to go for one of the bigger ecosystems where every problem I will run into has already been solved. That leaves ASP.NET (the sharpest knife in my block), Java (open source alternative), RoR, or Django. Django and RoR are similar, so I chose the bigger ecosystem with RoR. I was kind of sad since I like Python a lot better than Ruby. ASP.NET and Java are fabled to more enterprise centric, so I landed with RoR, where the startup community is much more vibrant.

On my next post, I will go into why this was such a difficult decision for me. Mainly because the only reason I chose RoR is because of the community, while disagreeing with every other reason the community have for choosing RoR.

ASP.NET Developer Meets Drupal

I've been working as a full-stack ASP.NET developer for the 2 years I've been out of school. I messed with Drupal a few years back as we evaluated a CMS at church. I thought it was a cool system to help non-programmers add content.

A few weeks ago, I met Chris Winn who gave me an unparalleled opportunity to take Kidbusy to the next level. Kidbusy is currently implemented in Drupal, so I am learning more about it. I've always thought myself to be pretty pragmatic, using the right tool for the job, and here is where my beliefs get tested. Drupal is a quick way to get started, but we all know that PHP is a difficult language to grow in.

I'm going through some tutorials at buildamodule.com, and it's been enlightening. After designing my own database schemas, I now see Drupal in a different light. It's basically doing the same thing I would've otherwise done in C# and SQL, but with a lot of wizards. I haven't dug into the database, yet, but I see adding content types as adding a row in the NodeType table. Adding fields are the same as adding a new column. Maybe it's implemented as multiple tables that inherit from each other. Maybe they're implemented as a single table. Hopefully it's not all in a postmeta table.

One of the major differences is that Drupal gives me a lot of stuff for free. I might've needed to spend a few days programming something, but Drupal will include it. I see this as good and bad. On the one hand, I get features for free. On the other hand, I have a lot more attack surface, maybe some I didn't think should be included.

Anyway, I look forward to this new experience. Maybe Drupal, not G, is the future of visual programming.

VB.NET in a Year

I've been working with VB.NET for a little over a year now. When I first started my job, I didn't know or care what language they used. I was quite surprised to see a software shop use VB. I thought that stuff had died a long time ago. Well, after a year, I think I've learned most of the language itself. There's still much to learn about the surrounding systems, but I'm pretty comfortable with the core of the language.

What have I learned about VB.NET? It's not really that bad. Let me go over a few pain points and then explain what I like about it.

Pain #1: No extension methods on objects of type Object. This is very annoying. We use ADO.NET extensively, and those things are not typed. If I request an INT NULL from the database, I want it back as a Nullable(Of Integer). However, it comes back as an Object which might be DBNull.

That's not a major problem. I can write an extension method that will do the conversion for me. Well, it turns out I can't. VB.NET just won't apply extension methods to Object. C# will do it just fine, but VB.NET won't. As a result, I can't use an extension method. I have to call the function on this thing instead of treating it as a function of object.

Pain #2: Really long closure syntax. I use the method form of LINQ extensively, so it's really annoying when 1/10 of my code is "Function (...)". C# makes it nice and easy. Who decided that VB.NET has to be wordy?

Pain #3: Tuple support. One of my favorite languages is Python. If I want to pass back two objects, I can pass it back as a tuple and assign it to a tuple. In VB.NET, I need to create a class or modify my parameters. I really don't like modifying parameters, and I am too lazy to create types for everything.

Anonymous types work pretty well for closures, but they don't work across function boundaries. If I return it, it becomes an Object in the caller. What happened? The compiler knew it had two fields before, how come it forgot it?

Say I do manage to get it back. How do I assign the fields to my local variables? I miss "(a, b) = (b, a)".

Pain #4: Generic Constructors. Say I have a generic class where the type is determined by the value passed into the constructor. In the new statement, I have to specify the type of the class even though the compiler can infer it from my argument. This is all good until I have to pass in an IDictionary(Of String, IDictionary(Of Integer, Integer)). That's way too much typing for something the compiler should do for me.

Pain #5: No dedicated dictionary syntax. I really want x = {a: 3, b: 4}.

Pain #6: (New Foo).Bar doesn't work. The compiler forces me to assign New Foo to a variable before I can call Bar on it. very annoying. I don't need to keep it. I don't want to assign it to a variable.

Pain #7: No macros. I want at least C style macros if I can't have Lisp style macros. I get neither. Awesome.

After a year, certainly there are things I like about VB.NET. What do I like about VB.NET?

Plus #1: IntelliSense. This is more about Visual Studio than VB.NET, but Visual Studio is one of the best parts about VB.NET. Not having to remember property/method names is awesome. Not having to remember the methods of types is also amazing. What can I do with this thing? Type '.' and IntelliSense will tell you right there. And it's always right. It never gives extra suggestions nor misses anything you haven't used before.

Plus #2: Edit and Continue. Even though I can't edit and continue anonymous methods, it's still magic.

Plus #3: Optional parentheses. I used to think this was really weird, but now I like it. This makes me type AddressOf often, but I think the parentheses I've saved makes up for the AddressOf that I have to type once in a long while.

Plus #4: CLR library. There's a lot of good stuff built in already. No need to reinvent the wheel.

That's about it. In conclusion, the only thing that VB.NET has over C# is Plus #3. But as I look back over this list, most of the stuff I don't like only cost me a few seconds more of typing. Seeing as how I don't spend that much time coding anyway, it's actually not that bad. If I had a choice, would I choose to use VB.NET? Probably not, but it's not as bad as I had first thought.

Initial Responses to XP

Most of our development team has been introduced to XP now. We agree that there are a lot of changes that need to be made. Some of these changes are internal to the team, and some require external changes. The two areas that we think need to start first are the XP team and short stories. Our existing team is both a huge team and no team. We have about 15 developers, a few testers, a few product managers, a few account managers who act as customers, and a project manager. The people are very friendly, and everyone helps each other in whatever way possible. The environment is very good for a team, however, team is pretty spread out physically and logically. I am a developer, so I am most familiar with the developer's perspective. It takes a while to talk to the testers or product managers, and it is nearly impossible to get the attention of customers in a timely manner. Many developers are very busy with their stuff, so it is hard to get help when I need it. I often have to wait several hours to get answers. No only is the team busy with stuff, everyone is busy with different stuff. We strive for collective code ownership by having everyone work on various portions of everything. There is a significant amount of code, and it takes a new developer weeks if not months to know enough about the code to be productive. In an effort to improve our bus factor, we assign developers to various portions of the system that they are unfamiliar with. The main problem with this is that they are individually assigned to a half-year project with little involvement from other developers. At the end of the half year, the developer is familiar with the code, but no one else is. So instead of whole projects having a single expert, we have many portions of projects having a single expert. We haven't really improved our bus factor much. To address these issues, we are attempting to have smaller and tighter knit teams. We will have testers and customers be more involved in the development process to reduce the amount of time it takes to get answers. We will try pair programming to improve the collective code ownership. Personally, I was expecting a lot more resistance against pair programming, but most programmers don't mind it as much as I had thought. We have been trying it for a few smaller sessions and it seems to work pretty well. Surprisingly, this is probably going to happen first before other improvements. It is likely to grow organically as well. While we get the team together, we will also tackle the problem of slipping deadlines. We came up with a long list of things to do to fix them, but the best thing we are looking at is just to create smaller stories that give us more milestones. We are going to divide up the 2-12 week stories that we currently have into smaller 1-3 day stories that we will be able to better manage. So all in all, we are hitting quite a few bumps in the road to XP, but we're excited about what kind of changes we can make to our process.

SyncLock Starvation

We had an issue in production that was particularly hard to track down. One of our applications appeared to be hanging while not doing a whole lot of work. We figured out that it was running a bunch of queries and hanging there. Looking at SQL Server, we saw that the connection was always waiting on async network IO, but the wait time was always pretty short (less than 2 seconds). We saw our application hang for hours at a time.

The funny thing was that this only happened occasionally, and without a good pattern. I had debugged the application several times, but it never had this issue while debugging. Finally, I was able to debug the application while it was hanging. Pausing all the threads, I noticed that they were all waiting for a single SyncLock.

A little background for this SyncLock: we have a bunch of databases that change frequently, so we store the database locations in one central database. To avoid hitting this database all the time, we cache the result in the application's memory. Whenever the application needs a connection string, it would first try the cache. If it's not there, then it goes back to the database to reload the cache. While the cache is being refreshed, it can't serve other threads looking for a connection string, so there's a SyncLock guarding this section.

This works great when we expect all our requests to exist. However, we recently added a new algorithm that checks to see if a connection string exists or not. Most of the time, it does not. This means that the application is constantly refreshing its cache. While it's refreshing, no other threads can access any databases, because the code that gets a connection string is guarded by that SyncLock. Thus, it looks like our application is hanging. The application would've eventually finished its job, but it would've taken a long long time.

This made me curious. Does SyncLock not serve requests in FIFO order? Can a thread be starved while waiting for a SyncLock? The answer appears to be yes. Here's the code to reproduce this.


Unfortunately, this isn't a super reliable way of producing the situation. We do see, however, that the maximum starvation time occasionally is much larger than it should be. Most of the time the threads are served in near FIFO order, so our application didn't always have an issue. However, sometimes, the threads are served in some other ordering, so our application did appear to be waiting forever for that lock.