This post is primarily a brain dump to aid my failing memory, and it was triggered by an issue with my Redmine Github Hook plugin where STDERR messages were not being logged.

The goal of this exercise is basically to figure out how to run a command and capture all its output - both STDOUT and STDERR - so that the output can be used in the calling script.

The test script I’ll be running basically outputs two lines, one on STDOUT, the other on STDERR:

#!/usr/bin/env ruby
puts "out"
STDERR.puts "error"

Kernel#` (backticks)

Returns the standard output of running cmd in a subshell. The built-in syntax %x{…} uses this method. Sets $? to the process status.

>> `./err.rb`
err
=> "out\n"

• STDERR is output, but not captured
• STDOUT is captured

Kernel#exec

Replaces the current process by running the given external command.

>> exec('./err.rb')
out
err

• Both STDERR and STDOUT is output. They aren’t captured as your process has been replaced so there is nothing to capture the output to.

Kernel#system

Executes cmd in a subshell, returning true if the command was found and ran successfully, false otherwise. An error status is available in $?. The arguments are processed in the same way as for Kernel::exec.

>> system('./err.rb')
out
err
=> true

• Both STDERR and STDOUT is output, but not captured.

IO#popen

Runs the specified command string as a subprocess; the subprocess‘s standard input and output will be connected to the returned IO object.

>> output = IO.popen('./err.rb')
=> #<IO:0x1017511b8>
>> err
output.readlines
=> ["out\n"]

• STDOUT is captured in a nice IO object
• STDERR is output

Open3#popen3

Open stdin, stdout, and stderr streams and start external executable.

>> require 'open3'
=> true
>> stdin, stdout, stderr = Open3.popen3('./err.rb')
=> [#<IO:0x101769da8>, #<IO:0x101769d30>, #<IO:0x101769c68>]
>> stdout.readlines
=> ["out\n"]
>> stderr.readlines
=> ["err\n"]

• Both STDOUT and STDERR are captured into nice IO objects.

Alternative: Using ‘nix redirection

An alternative to Open#popen3 (terrible name) is using standard output redirection:

>> `./err.rb 2>&1`
=> “err\nout\n”

This gives you both STDOUT and STDERR in one big string, which might be perfectly fine if you don’t require the granular control that popen3 brings to the table.

I am guessing this method would work for IO#popen as well as for the backticks.

From the get go, I chose primarily web applications to run my business on. Also, it kind of makes sense because that’s what I am building, and there is no better way to learn about good and bad web applications than using them yourself.

Let me give you a rundown of the most important ones.

All source code must be version controlled, and I use git for pretty much everything. Github is a great companion, and various open source projects as well as private and client projects are hosted there.

Issue tracking/project management: Redmine

For all Substance Lab run projects I use my own, pimped out installation of Redmine. Redmine is a great open source project management web application and it’s a great fit for software projects. It comes with git integration and a ton of plugins makes it possible to customize it to my needs.

Invoicing: Blinksale

Blinksale is a sweet little invoicing app that I’ve been using ever since I started Substance Lab. I have clients here in Denmark and outside the EU, and I need the ability to invoice in different currencies. I also need fairly flexible VAT settings. Back in the days, Blinksale gave me all of that and I like its focus on invoicing and nothing else.

Office apps: Google Apps for domains

I’ve been using Google Apps for email and calendars for a long time and I am pretty happy with it. For some reason, I have never really warmed up to using the Document editor part of the Google Apps and I keep NeoOffice around for those. I do use the Spreadsheet editor a lot and feel totally business managery when I do.

As an aside, “substancelab.com” is run on Google App Engine (mainly for kicks) and is powered by Webby.

Yes, Google owns me.

And then there are the desktop apps

Some stuff is still run locally on my machine, obviously. Code is written in TextMate, documents are backed up and shared via Dropbox, the full harddrive is backed up locally using SuperDuper! remotely using Mozy, ears are being pleased via http://last.fm and meetings are held via GoToMeeting.

What amazing apps are you using that I absolutely must use?

I have always promised myself that I would not fall prey to the blogging-fever that has been going around, and for a long time the title of my website actually was “This is not a weblog”…

Well, nothing lasts forever - neither did my opinion on weblogs. Welcome to my journal.

8 years later, I am still going strong (albeit not as strong as 4 years ago) who’d have thunk?

Holy time machine, blogman

Back then, mentalized.net looked like this:

So very, very blog-like :)

As much of this is still Webkit-specific it isn’t really something I get to use much on client work, but not being one to shy away from new, shiny things, I though it would be fun to dig into this.

I decided to recreate an interface familiar to Xbox players, namely the Xbox Dashboard. It seemed like a good candidate to try out techniques like CSS animations, font-face, multiple backgrounds, gradients, RGBA colors, drop shadows, and custom scrollbars to name a few.

The original inspiration for the actual look and feel is the Braid theme released by the great guys behind Braid - I hope they’ll forgive me for using their graphics.

The original theme looks like:

My recreated version looks like:

See the working demo here - remember, Webkit browsers only, so use Safari or Chrome.

I have tried to be generous with the comments in the source, feel free to dig in and see how things are made. Hopefully this can serve as inspiration/education for someone as it did for me.

Boo, you’re using Javascript

I tried briefly to make this work without Javascript using :target pseudo-selectors, but I never got it working. It might be possible to implement that, but my CSS-fu (or patience) wasn’t up for the task.

All the Javascript is doing is adding classes to the panels - .frontmost if it’s the active panel, .outside if it’s a panel that shouldn’t be visible. All animations and transformation is done in CSS.

It should be possible to create the behavior in Javascript instead of CSS. It would be interesting to see, if for no other reason then to see it running in browsers outside of Webkit. PDI.

Issues

Performance

The framerate in Safari is pretty bad. The framerate in mobile Safari on my iPod Touch is ghastly (and I totally miss not being able to swipe there). Chrome delivers the best performance by far.

Visual artifacts

When navigating between the panels and the animation is in progress, Safari doesn’t render the custom scrollbars. Instead it renders a transparent box there for some reason. Chrome handles that correctly.

I tried to add the reflections that’s visible on the actual Xbox Dashboard, but it simply caused too many visual artifacts in Safari. I’m guessing the combination of border-radius, scale, and rounding errors was tripping up the -webkit-reflection.

Should this ever be used in production?

Features that are only supported by browsers used by 10% of the internet-browsing population? I think not.

It definitely could be interesting to see how well this could be made to work in other browsers. Most modern browsers aren’t far behind Safari, and it might be possible to make it degrade gracefully enough in Internet Explorer. That is, however, far outside the scope of my experiment.

Today marks the first beta release of the coming version of Rails, version 3.0 and I figured it’d be fun revisiting that post.

1. $ rails hello

2. $ cd hello

3. $ rails generate controller hello

4. Open the file config/routes.rb. Almost at the bottom (line #57) is this line:

   # match ':controller(/:action(/:id(.:format)))'

   Remove the # in front so the line looks like:

   match ':controller(/:action(/:id(.:format)))'

5. Create a file named index.html.erb in app/views/hello containing the text 'Hello world'.

6. $ rails server

7. Navigate to http://localhost:3000/hello in your browser and be greeted by your friendly application: "Hello world".

PS: http://localhost:3000/ is (still) a lot prettier “Hello world”-ish page.

PPS: Note that the Rails 3 example above actually has a step more than the Rails 1 ditto. This comes from the fact that Rails 3 doesn’t build a default route for you.

It’s one of the best decisions I’ve ever made.

Being directly responsible for my own income has been fun, empowering, fulfilling, and pretty damn scary. I’ve met lots of great people, worked with great customers on great projects, and I’ve learned a lot of lessons in the process. Allow me to share a few of those lessons with you.

Your network is important

I am a geek. I am not particularly outgoing. Small talk is not a forté of mine.

However, as a freelancer, entrepreneur, or anyone running a business you cannot isolate yourself in a dark basement producing high quality work. You need to get out of your chair and meet people - partly in order to stay sane and get outside influences, but definitely also because your network is very important to your business.

I have gotten the majority of my customers through my network. If you provide a great service, your previous customers will happily recommend you and your coworkers will happily refer clients you. If you’re a great person, your friends are going to talk about you.

Leverage your network, and don’t forget to make yourself available for them as well.

Running a business will consume you

One of the things I have been most surprised by, is how much I think about the company.

There are always a ton of things to think about; processes that could be improved, ideas that might be worth pursuing, people to get in touch with.

Whenever I have some downtime; doing the dishes, walking the dog, watching television or in some other way allow my mind to wander, it tends to wander towards Substance Lab.

You won’t bill out as many hours as you think

Many other people have written about this point in length, but it is important enough to bear repeating.

If you plan on working reasonable hours - and you should - you cannot expect to work 8 billable hours per day. There are so many other tasks that you need to take care of, that a part of running the business. Admin, taxes, sales, marketing - Jon Hicks have a longer list.

I have, on rare occasions been able to work 6-7 billable hours in a day, but more realistically, I count on around 3-4 billable hours on any given day.

Scheduling is really hard

Things will take longer than you think. Period. Couple that with the fact that you probably won’t be able to work as many hours as you figure, and you have a scheduling problem on your hands.

You want to minimize the amount of time without client work. But at the same time you need to leave buffers in place for when your estimates are flat out wrong or when something goes wrong and needs to be dealt with.

I wish I had any words of wisdom to dish out that would instantly transform you - well, myself - into a scheduling demi-god. Unfortunately, I don’t. For now, I am relying on optimism and a lot of running really fast when things get tight - hardly optimal.

I’ve shown you mine…

That’s just a few of the lessons I’ve learned over the last years of freelancing. If you do any freelancing or run your own business, what have you learned? What would you tell someone who’s thinking about starting out on their own?

>> person = Person.first
=> #<Person id: 1, name: "Jakob", height: 170, created_at: "2010-01-22 08:57:02", updated_at: "2010-01-22 08:58:57">
>> person.height = 169,5
=> [169, 5]
>> person.height
=> 1

Uhm, huh? The above is from Rails 2.3.5 using SQLite, but I originally saw the issue running on PostgreSQL.

It turns out, that if the object you assign to an integer attribute/column does not respond to to_i, the attribute will be set to 1 - unless the object evaluates to false in which case the attribute will be set to 0.

My best guess is that ActiveRecord is trying to work around the lack of boolean columns in MySQL with this. So even though I am not using MySQL I still have to suffer its ineptitude. One just can’t win.

The concept is simple, started by Tim van Damme:

A simple game: Go to the address bar in your favorite browser, and type one letter. Start with “a”, end with “z”.

• A: apidock.com/rails
• B: Internal blinksale.com site
• C: Client Basecamp site
• D: Client Redmine site
• E: en.wikipedia.org
• F: facebook.com
• G: github.com
• H: Internal Highrise site
• I: images.google.com
• J: My Backpack site
• K: kongregate.com
• L: Client Lighthouse site
• M: mentalized.net
• N: Client staging site
• O: kongregate.com
• P: planetstorm.com
• Q: quotagio.us
• R: wiki.railsbridge.org
• S: substancelab.com
• T: twitter.com/mentalizer
• U: upcoming.yahoo.com
• V: Client staging site
• W: Internal Redmine site
• X: live.xbox.com
• Y: youtube.com
• Z: www.eizo.dk

Funny how some sites I’ve rarely visited still appear. Neither live.xbox.com, eizo.dk, nor quotagio.us are particularly common destinations for me, but I guess not many domains have z, x, or q in them.

You are in flow.

Suddenly an unexpected idea spawns in your head. This idea must be preserved and dealt with, but you can’t risk losing your flow, so you quickly launch your favorite notes application to jot the idea down before it escapes the grasp of your mind.

A new version of UberNotesMaster is available, would you like to download it now?

Stumped, you try to gauge whether the update is quick enough that you won’t lose your flow, but you have already lost. Your flow is interrupted, the idea is escaping, and your application is still not updated.

Do you prefer the plague or cholera?

Typically, when an application announces that it has a new version available, you have two options:

• Upgrade now and generally disrupt what you are doing
• Ignore and continue to run outdated, potentially buggy and insecure software, and get annoyed the next time you launch the application.

While I like upgrading my software as much as the next person, I launch an application for a reason; to use it - instead the application forces me to choose between performing the task at hand or get the latest version of the software. You know what, I want both!

Possible solutions

Update after use

Instead of asking me to update an application when I am actually trying to use it, how about dealing with the update after I am is done using the application?

This way, the upgrade dialog could appear when I quit the application, and the update would take place while I am doing my work in another application. A much smoother experience whose only interruption is easily dealt with.

It might, however, prove to be quite frustrating when you attempt to shut down your computer and all running applications want to update themselves. Also, OS X users are known to never actually shut down applications and also never restart their machines, so they would end up never updating.

Background update

A variation of the above would be to still ask me, the user, when I launch the application, but allow me to say “Yes, I want to update, please download and update in the background while I continue with my task at hand”. The actual update then happens after I quit the application, making the entire process almost invisible to me.

Out of band updates

We could also move the application update process out of the actual applications themselves. This strategy is known from the iTunes App store, where updates to all installed applications happen in the background while you’re doing other stuff. Smooth and seamless and allows you to maintain your installed software when /you/ are ready for it.

The process in itself is probably the smoothest offered, but it also bears the penalty of centralization. This software updater application somehow needs to know about all independent and unique applications you have installed and be able to update them.

Larger software vendors like Adobe have taken this approach and created their own - usually pretty crappy - updater applications. I would love to see something kickass here - preferably from the ISV community, but it could also be interesting seeing Apple build this into their own Software Updater app (or in the Windows equivalent if one exists).

Now, excuse me, I have to update my blogging application…

The decline of blog readers

The number of visitors to mentalized.net keeps declining. This is hardly surprising; I don’t write as much as I used to here (I have a feeling Twitter is eating up a bunch of potential articles), and a lot of content is probably being consumed off the site in news readers and other aggregators.

Still, I am quite happy having more than 125000 people visit my humble website and I owe each one of you my thanks.

What people came for

My collection of totally free, animated GIFs for showing Ajax activity continues to reign supreme as the most popular content on mentalized.net. It is followed by my satirical web 2.0 webdesign tutorial.

Both of those were published back in 2005 over 4 years ago, yet they still continue to draw readers to the website. The only article on the content top 10 that’s actually published in 2009 is my response to a Java developer spreading some FUD (which unfortunately detoriated into more FUD and mud-slinging).

Top content on mentalized.net in 2009

1. AJAX Activity indicators
2. Building your very own web 2.0 layout
3. No such file to load — mkmf
4. Java kicks Ruby in the what now?
5. Hello Ruby on Rails World
6. Rails 2.0 deprecations
7. Browser size does matter - Actual numbers
8. Hunting down VBScript error codes
9. Setting the request content type in Rails
10. AJAX activity indicators

How people got here

Google is still, hands down, the biggest source of traffic for mentalized.net. Almost half of all visits have been referred by Google. For comparison, the next search engine on the list is Bing with less than 1% of Googles referrals.

Social media sites like Digg, Twitter, and Facebook still aren’t providing me with a lot of traffic. I must be “old media” - or perhaps my content simply isn’t the kind of content that cries out for immediate attention.

The only post I’ve made in 2009 that was in relation to “current events” (link) did in fact get some traction on the social sites with roughly half the visitors coming from various social sites.

2010?

I am looking forward to redoing the above stats for mentalized.net when 2010 is over. I signed up for Project52 fully intent on accomplishing the goal (“to write at least 1 new article per week for 1 year”). Hopefully, getting back into the thrill of writing regular content should provide for better content that can at least break the decline of visits. Time will tell.