Browser Bubble, Part 2

For the next part in the explanation of my feelings about browser-based software, I’ll offer up a real world example that I’ve personally dealt with, blogging.

There is a good chance you’re viewing this post in a browser, and there’s nothing wrong with that. In fact, unless you read this blog all the time and subscribe to the RSS feed, it’s the only way to do it. It wouldn’t make much sense for me to write a custom application for you to read this blog; it wouldn’t even make sense for Blogger to offer one. The web browser was specifically designed for this type of duty. I won’t belabor that point, since I doubt I’d find almost anyone who would disagree, but I just wanted to make it clear I’m not a lunatic, and I do know the browser is very good for some tasks.

On the other hand, there are two parts to a blog. You get to read it, and throw back some comments if I’m lucky, but there’s also the part where I write it, create the templates, etc. And this is where the overreliance upon the browser platform gets frustrating.

Blogger provides a browser based editor for blogging, much like other online blogging tools. Currently, Google has a new version in beta of the entire Blogger system. One of the biggest changes in the new system is a new layout system. The new system uses a lot of AJAX and a web based widget system to allow you to customize your layout. There are a lot of improvements, like the ability to categorize posts, and a more flexible archive control, both of which you can see and example of right here on this blog.

Despite all these improvements, there is one thing I really dislike about the new system. The problem is the new design locks you into Blogger’s AJAX based layout editor. It used to be you could create your template in a WYSIWYG CSS enabled HTML editor, and then upload it to Blogger. You still can, except the problem is, the new Blogger templates use lots of proprietary code that just doesn’t work in any HTML editor I know of.

And while the new AJAX editor is better than the old textbox, it’s still extremely limiting. For pretty much anything non-trivial, you’ll have to edit the code manually. To add insult to injury, the Blogger area to edit this template is really tiny, and isn’t anything more than a dumb textbox. And because it’s part of a web form, you could spend a lot of time editing it, just to be presented with a timeout or some other browser based failure.

So, I usually take the contents and copy-and-paste them out to a real editor. It used to be a HTML editor, but now it’s just a text editor, which means that if I want to see the results I have to copy-and-paste them back to Bloggers little text box, and hit preview. You can’t imagine how frustrating that process, especially when you’re trying to figure out why the sidebar is indented 40 pixels more in FireFox than IE.

That annoyance is a result of poorly specified HTML rules (although if you’re keeping score it turns out that IE uses margins as W3C later recommended, whereas FireFox uses padding). But HTML standards can be fixed, with time. Also, HTML standards are a problem regardless of what I’m using to create them, because in the end I want to be able to make my blog available to people with browsers.

What is more important, are the insane hoops the Blogger system forces me to jump through to get a halfway decent editing experience. The root cause is the lack of file system type integration. If Blogger allowed me to access my template with something like WebDAV, then I could edit it directly. Instead, they make an even more locked in system.

Also, consider the difference between the Blogger template editing experience and that of a desktop app that can validate the XML, invoke IE, FireFox, etc, or even provide inline previews of either or both browsers. It’s hard to emulate FireFox inside a IE browser window, or vice versa. And despite all the effort Google has put into their AJAX editor, it’s not even an attempt at WYSIWYG. I’m sure it’s something that can be done, but it’s a massive effort, and I just don’t see the point in all that effort when Google could just release a desktop app to edit the layouts. That app could execute a HTML editor of my choice, and pass out the editable sections of the template, and all kinds of other nifty functionality that would be difficult to impossible using AJAX.

If I’m going to write 90+ posts, I’m certainly willing to install a single app. But the dogmatic view that the browser is the solution to all problems kills this option.

Luckily, a bit more wisdom has prevailed in the area of posting, although it is still neglected. For posting, my current process is to load up Word and begin writing. I’m using Word 2007, and was using it to post, until Google broke that, so now I copy and paste out of Word into Windows Live Writer to post. Some shorter pieces I may write directly in Windows Live Writer.

Why do I use copy to Windows Live Writer rather than the Blogger edit post function? Two reasons. One is that when you copy and paste from Word to the Blogger post editor you’ll often get bad formatting. With Windows Live Writer, it just works. I don’t know whom to blame for this, but posting with IE or FireFox, both have the same problem.

More importantly, however, if you copy and paste to the Blogger post editor, you don’t actually know if it’s going to work until you post. There’s a preview button in the Blogger post editor, but it actually looks nothing like my site. It’s somewhat confusing what the purpose of that pseudo-preview is. Windows Live Writer, however, shows me exactly what my post will look like before I post it. Not only can I see what the post I’m submitting looks like, I can see what it’s going to look like on my home page, with all the navbars and other posts.

There are a few flaws in this system. There is the annoying copy-and-paste from Word to Windows Live Writer, but the only reason for that is that I like having Word’s spell/grammar checker. Windows Live Writer does spell checking, but not in the same fashion as Word, and has no grammar support at all. The other flaw is that Blogger does not allow Windows Live Writer to use the label system. I’ve used categories from Windows Live Writer with other blogging software, but Blogger for some reason has decided to use a crippled blog posting API, or something. So after I submit a post, I have to log in to Blogger and edit the labels through edit post. I wonder what effects this has on RSS readers, as far as double posts, but so far my aggregator hasn’t ended up with doubles (I’m not counting those that showed up during the labeling of old posts I did a bit ago).

Now, you might draw different conclusions from this tale, but to me, the conclusion is that life would be much easier if the developers at Google who are working on Blogger would accept that editing posts, and certainly templates isn’t something that’s appropriate for a browser. They’d be much better off spending some time on a good desktop app to handle part of it, and enhancing the support for posting API’s, WebDAV, etc. Also, a good desktop app could easily integrate with other desktop apps in other ways. For one, it could allow open/save that doesn’t require 5 clicks, like the upload/download template buttons on Blogger do (actually for practical purposes, it’s a lot more than 5 because you need to navigate to the proper folder).

I think, for my next post, I’ll start to discuss some of the technologies available for smart client development on the Java and .NET platforms. But like I said in the first part, this is going to take a while, so hang in.

Browser Bubble

Since the original mosaic browser, the ubiquity, power and importance of the browser has trended in one direction, upward. Many feel this is merely a sign of things to come, but to me, I see it as a bubble ready to burst. There is no denying the role and importance of the browser, and I see no reason to expect it to shrivel up and die, but today, and for the immediate future, the browser as a platform has been leveraged not just beyond its original design, but beyond reasonable limitations.

Like most trends, the browser started by fulfilling a role that was genuinely lacking. The browser provided an easy to use portal into vast amounts of data waiting to be poured onto Web Servers worldwide. Some of browsers competitors (AOL, Prodigy, CompuServe) applied more control to the content in an attempt to create a consistent user experience, and provide greater interactivity. But the browser trumped all of that by allowing a simple brain dump of data. Much of the initial data on the internet was simply text, with sparse hyperlink additions. Interactivity was put on hold in the interests of developing quantity, and the web flourished.

Retailers, like always, flocked to where the customers were and began to establish online stores. The browser adapted well to the online store concept, because much like the initial round of web development, retailers had large quantities of mostly unstructured product information. This data had a great deal of graphics and formatted text, and there were the order processing systems to worry about. But in the large, there was very little interactivity in a web store. Click on stuff you want, enter credit card, poof. In most cases, the back end systems remained unchanged, and the browser handled only the customer view.

Providing a secure method of transferring information in the browser provided some hurdles, and still haunts some users today. Despite a few speed bumps, the browser eventually provided a connection to retailers that many users came to trust, at least a little.

For many users, this little bit of trust, developed into application phobia. To a degree, this is understandable, since an unknown application is certainly more dangerous than an unknown website, and until recently, applications provided few means to validate their authenticity. Today, however, platforms like Java and .NET can provide better guarantees of safety than the conglomeration of technologies that compose a complex web application. Unfortunately, neither platform yet provides great mechanisms for communicating these guarantees to potential application users. More importantly, even though there are some communication mechanisms, the average user does not yet understand them in the same way they understand their browsers.

That’s a big stumbling block for these platforms, but with time it can be resolved. More importantly, for many types of applications, such as those that users use every day, it just isn’t necessary. Only new applications scare users, not old ones. These types of applications also tend to be those where browsers perform the worst. Interactivity in your day to day life is simply a must.

AJAX is touted as the solution to interactivity on the browser platform. While AJAX is a great way to extend the browser platform, it’s not the panacea that it’s made out to be. It starts at a disadvantage by being built on top of the HTML/CSS layer that was never intended or designed to handle complex graphics, animations and all of the other elements that AJAX is being used for. As a result, speed, both of development and of execution, suffers.

Developers must learn a great deal of obscure details about several complete technologies, many of which overlap. Not only do they overlap, but the overlapping pieces are not cohesive in any sense. HTML, CSS, Javascript, Flash, etc. all require different tools. In the rare cases of tools that support one or more of these technologies, they are still treated as separate components or strategies.

All of this harms development speed for non-trivial applications. Trivial applications can benefit from the separated tools because if only one is needed, there is simply less to learn. But as you move into more complexity, the number of tools grows quite fast. Rather than simply learning existing tools in more depth, the web application developer will find they need to piece together a variety of tools.

As processor speeds continue to grow, speed of execution does continue to decline in importance, but most complex applications will still have a few key areas where performance is still a concern. Certainly today, the browser platform is incapable of achieving the same level of performance in these areas as a smart client, but it’s also likely that it will always be this way. A browser must be built on top of a client platform, which at the very least means it’s performance capability cannot exceed the performance capability of the underlying system. Performance capability doesn’t define actual performance, but a browser application is not just hampered by being built on top of one platform, but the need run on top of two layers of platforms, each with multiple implementations.

The first layer is of course the operating system. The browser platform can’t be specific to any operating system, which limits performance optimizations tremendously. The second layer is the browser itself, which is can be limiting not just in terms of performance, but functionality itself. I suppose it’s not entirely true that you can’t be specific to one browser, one platform, but if you are, you’re still limited by the way the browser chooses to hook together with the operating system. You should also begin to question what you’re really gaining from keeping the rest of your application browser based code. Are you really still writing a browser application at this point? Haven’t you lost one of the primary advantages of trust? And the secondary advantage of cross platform accessibility?

If these two concerns weren’t important in the end, why did you write a browser based application at all? Even if they were important, the browser isn’t the only way to get these capabilities.

I hate to stop a post at a point like this. I don’t feel like I’ve really proved anything yet, more just raised a lot of questions. But the question of these two platforms is a very complex one, and so unless I want to write the world’s longest blog post ever I’m going to have to break this up over time. For right now, I’m wondering how many others think the same way. Maybe I’m looking to stir up some controversy too, although I usually avoid that. Guess we’ll see.

Update: Part 2

Suggested Type System

I think I am in general agreement with the viewpoint that typing should be optional not mandatory (see Is Static Typing a Form of Bad Coupling? and Plugabble Type Systems)

The one problem I have with this view is that most people who hold it seem to think typing should be a rarely used performance kludge. I’d also agree that typing should rarely be used as a performance kludge, but I don’t think that means typing should be rarely used. There are other reasons to use typing. Actually, there are quite a large number of them.

For me, the main justification has always been safety. Typing also has benefit as a form of documentation. Safety and documentation are things you want most of the time, not rarely, so why would you rarely want types?

I think one of the problems is that most people look at typing as a way that framework designers can restrict their users. But what if typing didn’t restrict users, but instead guided them? I think this would still meet the needs of framework designers. A framework change doesn’t need to have zero breaking changes. All it really needs to do is cover the reasonable points of view. If you provide a safety net, and your users tear it down, I don’t think any (reasonable) person would fault the framework designers for breaking those users.

A Metaphor

Consider programming like some kind of stunt in an Indiana Jones movie. In this stunt, you’re walking across a narrow beam, with a big pit of spikes down below. There’s also poison darts wizzing by from little holes in the side walls. Today’s type systems however fit into a few categories.

On type are harnesses that keep you from falling off, but they slow you down, and make you an easier target for the darts. These are your mandatory static type systems. Another type are safety nets, that let you fall, but at least keep you from hitting the spikes. These are the dynamically typed systems. You could go without a safety net at all, and save a bit of money and setup time. This would be assembler. Or you could go with a cheap and poorly designed harness. It would slow you down even more, and it might break and let you fall. This would be C or C++.

Although this metaphor isn’t perfect, I’d consider the darts to be complexity. You might be able to take one or two since the poison is somewhat weak, but there wizzing all over the place, and they add up. The spikes of course are the core dump that happens when runtime typing isn’t employed. Many static type systems employ both the harness and net, because the harness has a release latch. But this latch (reflection), is somewhat awkward and hard to use.

In the end we realize the best way through is a set of shields (unit tests) to protect us from the darts. But the only thing available to build these shields with is grass, and it takes a lot of grass to build a shield, since grass is kind of flimsy, our shields can only absorb so many darts. Plus we still have to worry about falling off and having to climb back up from our net.

I probably way over extended that metaphor, but it’s kind of catchy, and has a nice visual, so I’m keeping it.

Resolution

What I’d like to see is a type system that helps keep us from falling off, but doesn’t slow us down. I’m not sure how to represent this in my metaphor. Maybe as some kind of lightweight harness with a quick release/attach mechanism. When you have it on your sure you won’t fall, but if you need to do something that the cables are too restrictive for, you can detach. It’s important however that the choice to detach is not something that happens by accident. But it shouldn’t be hard to use either.

I think I’m asking for a lot here, but I do think it’s doable. If we could agree this was the way to go, we could start designing the syntax behind it. In a sense, we have all the basic ideas we need. The JVM and CLR provide just about all the necessary capabilities. The problem is that reflection is ugly and hard to use. We need a better designed latch. Even more important, we need one that’s designed for users who have a shield strapped to each forearm.

I'm afraid I may have blown the argument with that long drawn out metaphor, so I'll probably write about this again, but it might be a fun to read anyhow, so I'll publish.. really.. I'm doing it right .. now!

Testing Ruby

I spent a fair bit of time over the last two months with Ruby. I actually started studying it about six months before, but I hadn’t really got to the point of writing anything non-trivial with it. At that point, it was more “theoretical”, than real. So two months ago I started doing some more complex stuff. For example, I wrote this little distributed abstract syntax tree interpreter. It’s nothing to brag about, and it’s probably completely useless, so I won’t bother explaining it, but the point was to understand Ruby a bit better.

As I expected, there were a number of areas where Ruby was beautifully brilliant and easy to work with. There were some disappointments however. That’s not really surprising either, since Ruby hasn’t been in the limelight all that long, and as a result many of the more expensive peripheral things like tools and libraries haven’t matured yet. And of course, like any language/platform, there are going to be room for improvement. There will be tradeoffs, things the community is working on, etc.

To be fair, I should probably start by listing all the great things about Ruby, but plenty of people who know a lot more about Ruby than me have already done that, so I doubt I’d really add much through that. Let’s just accept that I do believe there are great things.

That said I’m going to start taking potshots. Well, potshots is a bit harsh, but since I’m sure someone will feel like that’s what they are, we’ll just label them as such, and avoid all confusion.

IDEs

The first disappointment was IDEs. I started with Ruby in Steel, an add-in for Visual Studio .NET. The beginning was rocky as the first installers had some issues, but eventually I got it installed (and the newer installers are smoother). But I was immediately disappointed because I thought there was some rudimentary Intellisense support. There wasn’t however. The SapphireSteel guys are still working on that feature for their developer edition, and it sounds like it will be better than what I expected could be done, but we’ll have to wait and see. Oh, and the debugger was being troublesome.

The next up was RDT for Eclipse. Despite also advertising code completion, I found no signs of it. And once again the debugger was very troublesome.

I was about to give up on the idea of a debugger at all, when I found ArachnoRuby. It didn’t have or advertise Intellisense, but it did have a working debugger. Actually, it wasn’t just working, but designed for dynamic languages. For example, it allows you to attach a IRB console to a running program (after a breakpoint has occurred), and make modifications just like you had run your whole program in IRB.

Beyond that, another great thing was the Ruby Class Browser, which was far more useful than the standard documentation, or RDoc for finding classes and methods. Still however, there was no Intellisense, which I think would have been better because I got annoyed at having to swap windows to look something up.

So, it’s not all doom and gloom, but clearly, Ruby IDEs have a lot of room for improvement. And some features, like refactoring and Intellisense are much more difficult than they would be in a non-dynamic language. That’s one of those little tradeoffs. The smart dedicated Ruby people will probably just shrug their shoulders at those issues and move on about their work, accepting it for what it is, a tradeoff. The really brilliant and really dedicated Ruby people will find a way around 90% or 95% of the problem, although that will take time.

The idealists will probably respond that those features aren’t necessary, or are actually, despite common wisdom, are evil. Something along the lines of “Code Completion makes you stupid”, or “My Dual Core 3Ghz processor is too slow to run anything more complex than a text editor.” Oops, didn’t mean to start ranting like that… Moving on.

Unit Testing

Unit testing in Ruby is actually very good, despite the lack of any IDE support for it. Being dynamic is the biggest contributor to this. The array and hash primitives are also great for setting up test cases. But mostly it’s the dynamic nature of Ruby. Tests get virtually no benefits from static types.

Except the problem is, I found myself writing tests that would have been covered by a static type system. It’s not that the test cases would have been any easier to write with a static language, but there were plenty of them I wouldn’t have had to write at all. And even then, there were even more tests I didn’t write that would have been provided by default through a static language. And beyond this, in order to implement some tests, I had to put the equivalent of static typing into my code. It just ended up being a lot more verbose then simply declaring the type would have been.

For example, take this initializer:

def initialize(*exprList)
 @exprList = exprList
 @exprList = *@exprList if @exprList.is_a?(Array)
 raise "Values must be interpretable" unless @exprList.all? {|value|
    value.respond_to?(:interpret)}
end

(I know I probably screwed up a dozen things, so it’s worthless to nitpick my code quality. I’m an admitted Ruby Nuby) That last line is really the crux of the issue here. I expect some will feel it’s simply not necessary, but the truth is, it helped solve more than one bug when I combined it with unit testing. But that line is really just a really verbose form of List<Interpretable>.

So what we have here, is just another tradeoff right? Dynamic languages make unit tests much easier to write, but require more of them. But what if we started to treat static type information as a compact, concise and expressive form of unit tests, rather than as a straitjacket. Maybe if we did this, by making static languages that support dynamic capabilities, we could make unit testing easy, and avoid rolling our own static typing. Just an idea.

Summary

There is more to all of this then simply potshots at Ruby. The real point, is that we all have a lot to learn from each other. Here I’ve shown how the Ruby tool developers are learning from the Java/.NET tool developers, though obviously they have their own unique challenges AND opportunities as well. But likely, all of the hard work put into Ruby Intellisense will have some real value to static language tool designers as well, especially those working with type inference.

I’ve also shown how static language unit test tool developers can learn from Ruby unit testing. I’m going to assume there is probably a way to make the Ruby side of things more palatable as well, though I don’t know what it is yet. When you get criticism, look at it as an opportunity to grow, not an attack, even if it actually is an attack (this wasn’t).

Guilty

I admit it; right now, I'm an audience metrics freak in regards to my blog. I have a Google Analytics, and StatCounter tracker, and I watch my feedburner counts. In a sense, it's silly. There is some value there, but I know I'm blowing it way out of proportion. But I can't stop..

I wasn't always this way. When I started this blog, I just wanted to experiment with blogging, and was following the idea that writing would be an effective tool to extend my own thought patterns as well. If a person or two discovered it and liked it, all the better.

Overall, that went well. I think blogging has been great for extending myself, and it's been a nice outlet for many thoughts fit for public consumption. But after your write something, no matter what your original intentions were you start to feel the need to share it. That is as long as you don't think it sucked, in which case you're more likely to want to delete or revise it.

And so that led me to wonder, is anyone reading this stuff? What I found out was: no, no one was reading anything I wrote. And because I liked a few things I wrote, it was a bit annoying to see that not a single soul had read it. In a sense, that's worse than one person reading it and telling you it sucked, at least for me. At least a little criticism would push me back towards the delete side of the spectrum.

I had built it, and they weren't coming. And I could clearly see from the logs that it wasn't as if a few people dropped by and rejected me, quite simply, no one was showing up. I've probably been impatient about the whole thing, because this was only 3 months in, but I started trying to spread the word in other ways. And for this stage, metrics were really valuable. They gave me a good idea of what methods were working, and which were just pissing people off. I stopped doing some things, found some different avenues, and finally got some readers. It's not that many, but at least it's enough that I feel like if I did something really right someone would take notice, and then maybe they'd come.

But in the process, I developed the metrics habit. Hopefully I'll eventually kick the habit, but it's not that easy just yet. For one, I'm still afraid that small group will go poof and I'll be back where I started. I almost stopped doing the whole post categorization thing after I realized that it was repushing all my old stories. I was quite afraid that would drive all he RSS/ATOM readers away. I still don't know the result, but I thought, better now than later, so I pushed on and finished it up.

I have noticed a few things that help me ignore metrics. One is comments. Comments are far more valuable to me than any metrics, and so when I actually get them, especially if they are good ones, I hardly care about metrics. Another is volume. It's funny, but if I have an especially good day, like a record breaker, I'm more likely to ignore the metrics. I'm hoping this means that if I continue I'll eventually get to the point where I'm confident enough to simply ignore not just the metrics but also the whole idea of spreading the word in general.

But for now, I'm still guilty. At the very least, I can say I don't tailor my writing to raising metrics, or base it at all upon them. I wanted to say, that I don't tailor my writing to any audience, but in reflection, that's not entirely true. I do drop some controversial topics out of some posts when I don't want to detract from the main point. But that's not about metrics or readers, that's about not getting derailed into one of the perennial debates. These debates are not something you can resolve in a single post, or even a back and forth comment session. And because they almost always start by rehashing the same points over and over, I don’t care to waste my words upon them until I’m ready to dig into them deeply enough to hit something other than churned mud.

My Feed

I switched to the blogger beta not too long ago so that I could add tags to my articles. Unfortunately I think this reset my feed so you might have seen articles redownload. The new blogger feeds are much better, but I know how annoying it is to have a bunch of posts you already read show back up, so, I'm sorry.

Unfortunately, it's not done yet. I want to tag all my old posts, and that unfortunately requires me to go to each post, edit, and hit publish, which once again tosses them into the feed. I hope I don't scare everyone away ;p

Exceptionally unfortunate is that I don't have a ton of time to post so it's both quiet and noisy at the same time. Sorry again!

Community Dynamics and Emergent Systems

The Basics

At one point, there was a battle between digg and reddit for dominance in the socially filtered news arena. I’m sure that for the owners and employees of the two companies this battle continues, and Digg is clearly winning. However for the users of the two sites the debate has largely died out, and become a non-issue. Why? Well the simple answer is they are different communities, and to be honest the reddit users don’t really want the digg users at all. Reddit has, and probably always will have enough users to submit sufficient amounts of quality content to fill up the homepage. For many it’s now not a battle of getting enough quality in, but keeping the bad quality from rising too high, and thus obscuring the quality. At least that’s my take.

As far as my lean? I don’t like either much these days. I do like programming.reddit.com a lot. There’s also dzone.com, which doesn’t have enough depth yet, but has a good community and great design. The programming section of digg is atrocious however. General purpose news I usually go elsewhere for, but I find myself checking the digg main page sometimes. I’m not sure if it’s because of the story quality or just the cleaner look. Since it usually happens in periods of lower intellectual activity, I don’t really think about it. For me, going to digg is kinda like zoning out, so I can see why a nice pretty site would be an important factor then.

Don’t get those opinions wrong, I’m not trying to advocate anything with them. You may go for entertainment, or you may go for education. It’s not that hard to visit both sites, so I see no reason why you’d need someone else’s opinion on which is better. It takes maybe half an hour or less to make up your own mind, and have it fit your tastes.

Differences

What I’m really interested in however is the dynamics of the two communities. They’ve actually changed overtime for a number of reasons. Size is one reason, changes also occur due to site changes. Beyond the look and feel, two obvious differences between reddit and digg’s designs are friends and down votes. At reddit, every article has an up and a down next to it, and this has some interesting dynamics. At digg there is a “bury” option, but it’s separated in a way that discourages its use, and the way it’s treated in rankings is radically different.

I haven’t been able to fairly judge how reddit’s choice affects controversial topics. Common wisdom would suggest they would have a harder time rising to the top, but that’s just conjecture. What I do know changes is the importance of timing. The proportion of up/down votes depends on more than the quality of the article, it also depends upon mood. And at certain times of the day, or certain days of the week, people’s moods are better or worse. For an article destined for greatness this doesn’t matter much, but it actually can be a factor in the first half hour or so. A few early down votes can toss an article into the garbage and prevent it from getting any future attention.

The worst time is definitely in the late night. It would seem like that wouldn’t be a bad time because there’s actually a lot of user’s on these sites as that time, but the thing is, the users that stay up that late doing reddit stuff have probably been doing it a bit already earlier that night and become a bit more selective as the night progresses.

Friends

The second difference, the friends systems, has a surprisingly large impact. On reddit you can add friends, but the only effect is a little coloring of articles your friends rate. On digg, you get the same kind of feedback, but you also get channels including nothing but what your friends’ submissions and diggs.

I think this second feature gets a lot of use on digg, and it’s really changing the dynamics of digg. The majority of users on both sites spend most of their time on the “popular” or “hot” pages, and not a lot on the “new” and “upcoming” pages. I think however that digg as a percentage gets even less time from their users in the upcoming section. Part of this might be site design, but mostly I think it is because a lot of users have substituted their friends channel for upcoming. It may seem like a minor change, but the implications are huge. I believe digg is ruled by a fairly elite group of users with pretty much complete control over what makes it to popular. These users have a large number of friends who will take a look at anything they submit or digg.

It’s important to note that I’m not accusing anyone of gaming the system. These users’ friends aren’t some cabal set upon controlling digg, their actually organically grown sub communities. Unfortunately for the rest of digg, they control the whole site, and they are entrenched. Unless one of these users submits an article or diggs it, that article has about zero chance of reaching popular because there simply isn’t enough time or eyeballs in the upcoming queue to possibly get the number of diggs required for popular material. And of course, unless they submit articles that become popular they will never gain a friend base either.

So the status quo is here to stay until digg decides they actually want to change things to avoid this state. I’m actually a bit surprised they haven’t yet. It’s hard to imagine with the amount of visibility they have into the system, and all the number crunching I have to assume they do, that they haven’t figured this out yet. Of course, maybe they have and they just aren’t very troubled by it. It appears that a lot of digg users are perfectly happy with what this entrenched group is promoting. I’m not, but hey, I’ve been out voted, and that’s no big deal because there are other places I can go.

Systems in Systems

What is really interesting is how a strong self protecting hierarchal structure emerged by accident from within the intended chaos of a social filtering system. It’s almost a bit scary because it’s quite similar to AI, and doesn’t have any of the first of the “three laws”. I bet that sounded really out there, but my point isn’t that digg could take over the world, it’s that no one could have predicted the development of that self protecting structure, so why should we believe we’ll see the one that really matters coming.

Another interesting thought is that this AI like system is built on top of the intelligence of individuals, but those individuals did nothing intentional to create it. And despite being a core part of the collective, they don’t have any way within the system to attack the structure. That attack must come from the outside to have any success.

Good Ideas Taken Badly

It appears other people had a similar reaction to Steve Yegge's Agile rant. The funny thing is that as wrong as I think the basic argument is, it has a good core of thought. Somehow it went astray. It's a bit scary how thin that line really is. You know, how a set of good ideas and good principles can appear to be justifying a bad argument. That, ironically enough, is exactly what Yegge attacks Agile for, when he points out how the short cycles can be used to make a project more date focused, rather than less as Agile is intended to.

It is a risk, when you talk about short release cycles, that managers will treat them the same as the old release cycles, but expect them more often. But when properly used, an iteration is more of a goal than a make or break moment. Sure, you put out a build, but there are no guarantees any specific feature will be part of it. Whatever is done is what's done. And you use what you learned about your velocity to predict the future. But predictions should remain just that, predictions. Not commitments. And they aren't that way just because Agile says so. No, it's because the incremental releases relieve pressure and present clear alternatives.

High Tech Agile

But the possibility of misuse isn't his only criticism of Agile. He also attacks many of the "low-tech" solutions, such as pair programming, whole team, and story cards. These arguments actually have a ring of truth to them, because it does feel like there should be a better way. A way that doesn't require all of the trade-offs, or at least not as many as these introduce. But that feeling is irrelevant, at least in an immediate sense because these low-tech solutions are better than the pre-existing alternatives.

Yegge does offer up a high-tech alternative to story cards though, and as I've written it's very interesting. The only problem is it's inaccessible to most of us for the moment. I'm sure that will change, but in an immediate sense, the best options available are the low-tech variations. Thought into how to evolve these practices further through technology is certainly justified.

For example, I'm starting to believe the concept of an office will go the way of the dodo, at least for programmers. Despite its sensibility, sitting together has always bothered me because of the lack of privacy. Sure, you can provide private spaces, but then you have to switch spots, and deal with all the associated hassles. Plus, what if the team next door is just too loud? Or what about people with bad hygiene or simply annoying habits. That kind of environment is extremely fragile, and unless someone is intentionally obnoxious or inconsiderate, rather than just dealt an unfair deck (smelly, sneezy, etc.), it's hard to justify their removal from a moral or legal sense.

My solution is to use technology to create a virtual shared space. The beauty of this is that you can turn it on or off, and selectively include or exclude parts of the team. It's dynamic, and possibly best of all tackles one of the largest hurdles, the remote worker issue. Unfortunately the tech just isn't quite here yet. It's coming, no doubt, and pieces and parts are here today in various forms of quality and incompleteness, but it's not yet complete. And considering how difficult it is to argue the case for a second monitor for every developer, it's probable that when it is here, it will be difficult to argue the case for the proper equipment.

Pair programming is another piece that would benefit from this tech, and the greater dynamism. But pairing could also benefit from several other techniques that are doable today. Another blogger writes an interesting way of doing parallel pair programming (read the end). I've often thought about just plopping down a few extra monitors on a developer's desk dedicated to watching what his pair is doing, thus allowing them to collaborate without one of them being relegated to the stone ages of paper and pencil (or more commonly marker and plastic).

Back to the Point

Agile is definitely improvable, definitely full of flaws, but rejecting it in favor of the old way isn't right. I don't think this is what Yegge meant, but it's what more than a handful unfortunately read. It's not an uncommon problem either. We often make the mistake of attacking the solution to one problem, because it's not quite perfect, while ignoring the real problem. Why are people more concerned with rewriting Java programs when they have a heap of COBOL code lying around? Both have merit, but one is a far bigger problem. I'm sure part of the issue is that rewriting the Java code is less difficult, but COBOL code isn't going to update itself, and the number of people willing to even look at it is shrinking faster than the amount of deployed code.

Unfortunately there's no general solution to this problem, because you can't always say one end is right, and the other end is wrong. A few issues gradate evenly from black to white, but many others are a mixed bag. The best examples here I can think of are the static/dynamic debate, and the browser/client debate. Personally, I believe that static is the "right" way. But there are a lot of great things about dynamic languages like Ruby, and if I took a black and white view, I'd ignore that. Learning and using Ruby has opened me up to these advantages. But for every advantage so far I see a counter.

For example, unit testing is beautifully done because of the flexibility afforded by the dynamic nature. But on the other hand, I find some of the unit tests I'm writing implement a hand-coded custom version of static analysis. Not typing variable types can speed things up a bit, but I find I lose much or all of that to extra syntax debugging due to a misspelled variable name or such, and the problem that I may need several test cycles to find consecutive syntax errors, rather than one compile cycle (or a squiggly).

For this last part, syntax debugging, better Ruby IDE's can certainly help, but there isn't any reason a static language can't use type inference to remove 95% of the extra typing. Likewise for the first part, unit testing, there's no reason static languages couldn't make a special exception to type checking, encapsulation, etc. in unit tests. The JVM and CLR have all the necessary pieces and parts to support this. The only downside might be speed, but how fast your unit tests run is not your number one performance concern. (And a static language faking dynamism through reflection is still likely about the same speed as the natively dynamic language.

Clearly this is an area where both sides have a lot to learn from each other, and the best solution lies somewhere in between, not at one end or the other. My belief is that it will be easier to add dynamic features to static languages, than the other way around, but other than that, I see value in both arguments. My real point here is not whether static or dynamic is better, it is that a dynamic or a static language advocate both have a set of good arguments that can easily be used to justify the wrong conclusion. That conclusion is that everything the other side believes is wrong because static and dynamic are opposites and in their minds, wholly incompatible. It's not true, but it's a case where the truth is easily and often ignored.

Browsers and clients are another great example of this same problem. Since I'm not going to take the time to explain my belief of which is ultimately the "right" way, I'll simply direct you to a similar viewpoint. Curiously another entry from Yegge.