There are many good reasons to be concerned about whether the internet industry is in another bubble. Once shaky, unproven businesses hit the NASDAQ, the chance that granny is going to lose her pension fund goes up big time. That’s the direct economic consequence that you’ve probably heard already. What hasn’t been discussed as much are some of the secondary effects that a bubble has on the industry. Like how it gets harder to hire good people as a consequence. Sure, you may have heard of The War for Talent, but it’s much deeper than that. This is not simply about the latest tech darling not being able to find programmers in the Valley. When bubbles inflate as fast as the most recent one, support infrastructure can’t cope. There just aren’t enough programmers, designers, operations people, and other warm bodies to man all the hot air balloons. So you have a predictable effect: Rapidly increasing demand for an only steadily increasing supply. Thus, inflation. All this new demand is being fueled by the endless flows of cheap cash being pumped into the Valley (and everywhere else it can go). So the circle goes like this: And every time we do a round, more talent gets trapped in the bubble, chasing things that can be flipped for the most money. Now in theory this is all good and well. Resources are allocated to the pursuits with the most economic value. So if Instapintora is worth $2 billion, of course they deserve to be able to hire 1,000 web builders, right? That’s the same argument you’ll find for bridges to nowhere and other pursuits void of inherent economic and public value. Sure, people are being employeed, money is changing hands, but come Monday morning, the hangover is that we spent a bundle to build a lot of shit that’s not going anywhere. As a result, we missed out on doing other worthwhile things. All those smart and talented heads, and all those benjamins, didn’t progress the economic base in a way we’re going to care about tomorrow. And that’s a damn shame.

I was bouncing around the Rails API documentation yesterday, and I noticed a few rails console tricks I haven’t seen before. There’s been plenty of posts about irb and Rails before, but I’m hoping you’ll learn something new here. The following console samples were taken with Basecamp Next on Rails version 3.2.3. Dive into your app Running the app method in rails console gives you an integration session instance, so you can use it just like when you’re a normal integration test for Rails. >> app.class => ActionDispatch::Integration::Session Generating routes was always a hassle. What module did I need to include again? Did you remember to set the default_url_options? Stop googling and just use app: >> app.project_path(Project.first) => "/projects/130349783-with-attachments" It can make requests inside of your app as well: >> app.get "/735644780/projects/605816632-bcx.atom" => 200 >> app.response.body => "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n<feed xml:lang=\"en-US\" ... Check out ActionDispatch::Integration::Session and ActionDispatch::Integration::RequestHelpers to see what else this object can help you with. Try out a helper Getting a console session bootstrapped with Rails’ helpers is also a pain, which helper can fix for you! You could also use it to play with building HTML tags, or any existing Rails helper that ActionView knows about. >> helper.truncate("Testing", length: 4) => "T..." >> helper.link_to "Home", app.root_path => "<a href=\"/\">Home</a>" One gotcha with helper is using instance variables inside of helper methods. Yes, I’m aware this is a Bad Idea™ with helpers in general, but I haven’t worked in a single Rails app without one. Here’s a small example of a method with one: def title_tag(title = nil) if @project.present? && title.nil? content_tag :title, @project.name elsif @project.present? content_tag :title, "#: #" else content_tag :title, title end end We can use our old friend Object#instance_variable_set to violate a core principle of OOP and let us try this helper out in the console: >> helper.title_tag "Testing!" => "<title>Testing!</title>" >> helper.instance_variable_set :@project, Project.first => #<Project id: 130349783, ... >> helper.title_tag => "<title>With attachments!</title>" >> helper.title_tag "Posts" => "<title>With attachments!: Posts</title>" Dealing with a helper that uses params is also not straightforward. However, we can trick ActionView into listening to us with a tiny hack. This is the console, after all! Let’s say we have this helper method: def javascript_debugging_options if params[:javascript_debugging] == "enabled" { debug: true, digest: false } else {} end end Usually ActionView needs an entire ActionDispatch::Request from the controller to figure out what parameters came in from the user. We can trick it with a little OpenStruct instead: >> helper.controller = OpenStruct.new(params: {}) => #<OpenStruct params={}> >> helper.javascript_debugging_options => {} >> helper.controller = OpenStruct.new(params: {javascript_debugging: "enabled"}) => #<OpenStruct params=> >> helper.javascript_debugging_options => {:debug=>true, :digest=>false} Where does that method come from? Hunting down the exact location of a given method isn’t always easy. Luckily, Ruby can point you in the right direction with Method#source_location: >> Project.instance_method(:trash).source_location => ["/Users/qrush/37s/apps/bcx/app/models/project.rb", 90] Whoa! You’ll get back an array with the full path of where the method was defined along with that method’s line number in the file. I’ve used this when looking for code buried inside of gems as well. Let’s try it on app: >> app.method(:get).source_location => ["/Users/qrush/.rbenv/versions/1.9.3-p194/lib/ruby/gems/1.9.1/bundler/gems/rails-7d95b814583b/actionpack/lib/action_dispatch/testing/integration.rb", 32] This tip has saved me from an immense amount of source diving. Try it out! More tips Looking for more tips? JEG2 gave a neat talk at RailsConf about some of the hidden parts of Rails you might not know about. If you have some of your own, please share!

We’re looking for another teammate. This time we’re looking for someone who is completely focused on improving conversion and retention. You love moving the needle, one small step at a time. This job is all about seeing untapped potential. Conversion could be financial (get more people to start a trial or complete it – what we consider a “sale”), or outcome-based (inspire people to create more Basecamp projects by showing them creative ways to use Basecamp they never thought of before). Or something else. Retention could be reducing cancellations, or reducing unsubscribes from our mailing list, or increasing the number of people who log into Basecamp every day. Or something else. You should… You should be great at figuring out which words or phrases hook people instead of letting them get away. We believe the right words at the right time can make all the difference. You should look at a sign up form like… …and already know three ways to significantly improve conversion. You should be great at understanding what motivates people. This requires deep insight into the decision making process. You should know why people buy, why people don’t buy, why people buy more of something they already have, and why people who bought decide not to buy any more. You should look at a screen like… …and think to yourself – “if they rearranged this, highlighted that, or renamed this other thing, they’d increase sales by at least 25%.” You should be great at knowing how to help people find value in something they already have. There are always more ways to use something than the ways you already use it. You should look at a new project like… …and say “no, no, no… a better way to get someone started on a new project is like this…” Or you should look at an email like… …and think “You’re missing a big opportunity to get more people to take action here. If we…” Ideas and execution This is not just a job for an idea person – it’s about ideas and execution. You’ll be expected to write the copy. You’ll be expected to implement the designs. You’ll be expected to shepherd ideas from start to finish – not just to toss them out there for other people to worry about. You should love to propose, experiment, implement, execute, and measure. You may not be able to take everything on 100% yourself, but you try your best before you hit your limits and have to ask for help. The challenge: Double our sales We’d like to double sales over the next 12 months without making deep material changes to our products. We don’t believe it’s possible. Change our minds. Convert us. Can you do it? If this sounds like something you’re great at, we’d love to talk to you. This is a new position at 37signals, so you’ll be in charge of making it work. You’ll be part of the best overall team in the business. From ops, to programming, to design, to support, to data, to video – we’re ready to work together to see your bright ideas through. To apply Send an email titled [Conversion] to jointheteam@37signals.com telling us why you’re the one. How you tell us is up to you. Our office is based in Chicago, but our team is spread out all over the world. You can work from anywhere. We just require a few trips to Chicago a year when the whole company gets together. We’ll be accepting applications until June 4. This is a chance to do the best work of your career. We know you’re out there. Get in touch.

A common obstacle we face when releasing new features is making production schema changes in MySQL. Many new features require additional columns or indexes. Running an “ALTER TABLE” in MySQL to add the needed columns and indexes locks the table, hanging the application. We need a better solution. Option 1: Schema Change in Downtime This is the simplest option. Put the application into downtime and perform the schema change. It requires us to have the application down for the duration of the “ALTER TABLE”. We’ve successfully used this option for smaller tables that can be altered in seconds or minutes. However, for large tables the alter can take hours making it less than desirable. Option 2: Role Swap This is the option that we have used in the past to perform schema changes on large tables. There are quite a few steps which make this option error-prone. When the changes are made, a short downtime to swap the application between the master and replica is required. Here’s a sample of the process we follow to change the roles of the current master “A”, and the current replica “B”. On B, STOP SLAVE. On B, SHOW MASTER STATUS. This requires that binary logging is enabled on B. On B, execute schema changes with ALTER TABLE statements. On A, CHANGE MASTER TO MASTER_HOST=’B’, MASTER_USER=’...’, MASTER_PASSWORD=’...’, MASTER_LOG_FILE=’...’, MASTER_LOG_POS=nnn; according to the master status reported by B. On B, START SLAVE. Let B catch up to A. Stop applications writing to A, and watch SHOW PROCESSLIST until activity is totally quiet. On A, SET GLOBAL read_only=1; On B, STOP SLAVE once you’re sure it has applied all the changes replicated from A. On B, SET GLOBAL read_only=0; Reconfigure your applications to write to B instead of A, and restart applications. START SLAVE on A. Finally…Have a stiff drink. You need it at this point. Option 3: pt-online-schema-change We have recently started using pt-online-schema-change to perform our schema updates without needing to take downtime. It was developed by the folks at Percona. During the online schema change the original table is not locked and will continue to take reads and writes. In very basic terms here is what is happening when you run pt-online-schema-change. Create new table with same structure as original. Update schema on new table. Copy rows in batches from original table. Move original table out of the way and replace with new table. Drop old table. Copying all rows of a table to the new table creates a lot of data that needs to be replicated to your slave. Fortunately pt-online-schema-change will monitor the status of your slave and will pause the data copy process if replication goes too far behind. We have also found that the data copy process can be disk intensive and will impact MySQL’s performance. Even with pt-online-schema-change we still choose to perform schema updates after hours to limit the impact on our applications. There are a number of settings in pt-online-schema-change which you can tune to maximize performance during the schema change. We also thoroughly test each schema change in staging prior to running production. Next up: MySQL HA and Failover

We have customers around the world doing extraordinary things with our software, but Ben Saunders is taking it to a whole new level. Ben and his team are using Basecamp to organize an expedition to the South Pole and back, unsupported and on foot. This is the same journey Captain Robert Scott died trying to achieve 100 years ago, and no one has attempted it since. Ben has been a professional polar explorer for more than 10 years and is one of only three people to complete a solo journey to the North Pole. He will be joined by Alastair Humphreys, who has cycled 46,000 miles around the world, and renowned nature photographer Martin Hartley. You can follow the expedition, which launches this October, at Scott2012.org and be sure to follow @polarben on Twitter. Special thanks to Temujin Doran for letting us use his amazing arctic footage.

A year ago we put Sortfolio up for sale. We entertained a variety of offers, met with a few potential suitors in person, and negotiated numbers. Ultimately nothing came together. Then we shelved the process so we could focus all of our efforts on designing and building the new Basecamp. Sortfolio continued to run itself for another year, generating over $200,000 in profit for us during that time. Our paying customer count continues to hover consistently in the 170s, each paying $99/month to be listed as a premium member. The price We’ve put Sortfolio on the clock: We either sell it by July 1, or we close it down. And we’ll make it easier this time so there’s no guessing: The price is $480,000, cash. No special deals, no partial payments now the rest later, no equity in your company, etc. $480,000 cash and it’s yours. Considering how much hands-off free cash flow it generates, we think that’s a very fair price. We just want it to go to a good home. We’ve put next to no effort into it over the last year… If you put some effort in you should see significantly higher sales. With over 10,000 free accounts, there’s tons of untapped opportunity here in the existing customer base. And of course plenty more on the outside or in entirely different industries. What you get You get the design, the branding, the code (it’s a Rails app), the customers, and the steady cash flow. Since Sortfolio is hooked up to our centralized billing system, you’ll need to write your own code to charge customers. Also, since Sortfolio is hosted on our server cluster, no hardware is included (but you can run it on a cheap server or two – the hardware requirements are next to nothing). We’ll also be happy to announce the sale here on SvN and on our very well subscribed email newsletter. Interested? If you’re interested at $480,000, please email me direct at jason@37signals… Thanks.

Keynote by David from RailsConf 2012 on progress, curiosity, fear, and the danger of easy-bake ovens.

This is going to be a big one. Now you can post messages, to-do lists, files, and text documents to the all new Basecamp just by sending an email. First, some background In Basecamp Classic, you can post a new message by sending an email directly to Basecamp. This feature is especially useful if you’re on the road and want to post a message to the project from your phone. Just fire up your email app, address the email to Basecamp, and it’s immediately posted as a message to the project. Initially, the all new Basecamp didn’t launch with this feature. Like everything else in the new Basecamp, we wanted to approach it fresh, take our time to think it through, explore a variety of ideas, and make it even better than before. Better than before In Classic you could only post a message via email. You couldn’t make a to-do list, start a text document (called Writeboards in Classic), or upload a file via email. But in the all new Basecamp, you can do all these things and more. Plus, it’s easier, clearer, and more powerful than before. Here’s how it works At the bottom right corner of every project you’ll see a link that says “Email content to this project”. When you click that link, you’ll see a sheet pop up that looks like this: You’ll see five icons at the top of the page. Each icon describes a different email-in feature. Starting discussions, making to-do lists, creating documents, uploading files, and forwarding emails. Clicking any one of those icons shows you the simple steps to follow for each feature. Below the steps we show an example of an email and how that content will look when it’s posted to Basecamp. When Basecamp receives your email and posts your content to the project, it immediately sends you an email receipt letting you know it worked. For most content types, the email also includes a link to let you notify others on the project letting them know that you just added something to the project. If you click that link you’ll see: Since we expect a lot of people to email in from their mobile phones, we made a mobile optimized version of the notifications screen too: And a bonus feature In addition to allowing you to start discussions, make to-do lists, create text documents, and upload files via email, we also allow you to forward in any existing email to Basecamp. This is especially useful if there was a conversation going on outside of Basecamp. Just forward that email thread into Basecamp and it’s stored along with everything else on the project. Easy peasy. Forwarded emails get collected into a “Forwarded emails” section at the bottom of the project. And if the email was an HTML email, we show the full HTML email, too: So many uses There are so many interesting ways to use this new email-in feature in Basecamp. One of the things I’ve loved about it so far is that I can sketch an idea on the whiteboard/chalkboard in a meeting room in our office, take a picture of it with my iPhone, and email it directly to a Basecamp project. It’s such a great way to get the physical results of a brainstorm, meeting, or sketch session right into Basecamp. And now that it’s in Basecamp, I can erase the whiteboard and not worry about ever losing that idea. Email-in is also a great little “poor-man’s API”. You can have your apps send emails direct into Basecamp whenever you make a sale, get survey results from a customer, or who knows what. The opportunities are endless. We’re eager to see what sorts of things people come up with. Thanks again We’re thrilled with the feature and we think you’ll feel the same way. Thanks again for using Basecamp!

In college, I worked for a couple of years in a lab that tested the effectiveness of surgical treatments for ACL rupture using industrial robotics. Sometimes, the reconstructions didn’t hold. The surgeons involved were sometimes frustrated; it can be hard to look at data showing that something you did didn’t work. But for the scientists and engineers, all that mattered was that we’d followed our testing protocol and gathered some new data. I came to learn that this attitude is exactly what it takes to be a successful scientist over the long term and not merely a one-hit wonder. Occasionally, when we’re running an A/B test someone will ask me what I call “success” for a given test. My answer is perhaps a bit surprising to some: I don’t judge a test based on what feedback we might have gotten about it. I don’t judge a test based on what we think we might have learned about why a given variation performed. I don’t judge a test based on the improvement in conversion or any other quantitative measure. I only judge a test based on whether we designed and administered it properly. As an industry, we don’t yet have a complete analytical model of how people make decisions, so we can’t know in advance what variations will work. This means that there’s no shame in running variations that don’t improve conversion. We also lack any real ability to understand why a variation may have succeeded, so I don’t care much whether or not we understood the results at a deeper level. The only thing we can fully control is how we set up the experiment, and so I judge a test based on criteria like: Did we have clear segmentation of visitors into distinct variations? Did we have clear, measurable, quantitative outcomes linked to those segments? Did we determine our sample size using appropriate standards before we started running the test, and run the test as planned, not succumbing to a testing tool’s biased measure of significance? Can we run the test again and reproduce the results? Did we? This might sound a lot like the way a chemist evaluates an experiment about a new drug, and that’s not by accident. The way I look at running an A/B test is much the same as I did when I was working in that lab: if you run well-designed, carefully implemented experiments, the rest will take care of itself eventually. You might hit paydirt this time, or it might take 100 more tests, but all that matters is that you keep trying carefully. I evaluate the success of our overall A/B testing regimen based on whether it improves our overall performance, but not individual tests; individual tests are just one step along what we know will be a much longer road.

Today I saw a familiar pattern on a website for an analytics product. The home page pitched the product, and then above the signup form there was a headline: “Sign up in seconds.” I see this all the time on signup forms and it makes me wonder: why did the designer put that there? My best guess is that they were trying to relieve some anxiety the customer might have. Like, “don’t worry, it’ll be over soon!” I’ll bet that the time-to-signup isn’t an important anxiety factor. When’s the last time you shopped for a software product under intense time pressure, where every second counts? When I evaluate web products I often feel uncertain about what will happen after the quick signup. Sure it takes seconds to create an account, but then what? I had an idea to address this uncertainty. You could preview the workflow steps that come after the signup so it’s clear how much of a gap there is between signing up and getting value out of the product. Check out this sketch. It shows “what happens” after you signup. Once you sign up, you get a Javascript code, paste it into your website, and then you can watch real live graphics of traffic come to your website. Sounds pretty easy right? Why not try it? I haven’t tested this approach on any sites. Intuitively I like how it integrates the call to action with the sales pitch in a single flow. What do you think?

Over the past few months I’ve learned an immense amount from the all new Basecamp codebase. This year for RailsConf, I was able to show off some new (and old!) patterns in the codebase along with some neat libraries that we’ve put to serious use for our customers. The slides are over at SpeakerDeck if you’d like to see them all, and hopefully a video of the presentation will be up soon! Here’s a few choice sections from the talk, and plenty of great links to check out in the meantime: Production setup We’ve talked in bits and pieces about the infrastructure behind our apps, and I wanted to give a clearer picture of what Basecamp looks like both on our production machines and on each developer’s box. For databases, we hook up to several: MySQL: Basecamp’s best friend. Stores pretty much all application data. Memcached: Used for our key-based caching system. Our 1 TB cluster just started evicting keys last week! Redis: Used mostly for Resque, and for tracking live page updates with sorted sets. Elastic Search: Fantastic, open source searching. There’s a mix of internal services as well, which are hooked up to through a series of gems and Rails engines: Depot: Where your files are stored. I’m sure this is an old rails/train pun. Portfolio: In charge of displaying and resizing avatars across all of the services. Launchpad: Our single-sign-on gateway. Handles all authentication concerns. Queenbee: Where the honey’s made! This is our internal billing system that manages plans, accounts, trials, and more. Development setup Thankfully, we don’t need to set up each individual service on our own machine to get hacking on Basecamp. Two are required though, along with the main Rails app: Portfolio, and Launchpad. The app would look pretty silly without avatars, and even locally we need to sign in to get access. Running these services side by side is usually a challenge, but Pow handles it like a champ. If you haven’t checked it out yet, it’s a wonderful Ruby web server that hooks into OSX internals to make serving apps locally painless. We also have a convention across all of our apps to use a single bash script that makes getting started with development so much easier. This script lives at script/setup in each repo. Typically, getting an app working on your machine feels like blowing on an NES cartridge. Having one script to be a “reset button” for your app that sets up the database, bounces your local web server, and does whatever custom wrangling required to get cranking away can save an immense amount of time and frustration for your team. Be Concerned ActiveSupport’s Concerns are used extensively in our models and controllers. Using them in your Rails app enables two big wins: it’s the simplest refactor out there, an Extract Method, and it allows you to group alike methods together instead of each class being a grab bag of unrelated functions. Here’s an example of our Ajax concern, used across several controllers: # app/controllers/concerns/ajax.rb module Ajax extend ActiveSupport::Concern included do before_filter :set_ajax_cookies end private def set_ajax_cookies # set some cookies! end end It’s a standard Ruby module, but the included block that allows class-level code to be be easily added in. Of course, even if you extract everything into concerns, your class can still be a ball of mud. Please don’t think this is an excuse to gloss over proper design for your code. JavaScript Behaviors The Ruby to CoffeeScript ratio is around 1:1 as well. Since Rails 3.0, using HTML5 data-* attributes for decoupling behavior between the client and server side has been immensely helpful. We’ve taken that to the next level with our usage of data-behavior. Here’s an example of what hooking up the date picker for todos looks like: <label> <b>Set the due date:</b> <div data-behavior="date_picker"></div> </label> $('[data-behavior~=date_picker]').live 'focus', -> $(this).datepicker() This technique keeps complicated JavaScript logic out of the views, stops tying behavior to CSS classes and/or DOM IDs, and it’s still pretty easy to query the elements. Check out the usage of ~= as well, which allows us to chain multiple behaviors on the same element. If you’d like to get a start on using these in your app, rails-behaviors has some great examples. Lots of Logging Anything we can do to make debugging customer problems easier and faster is worth it. Noah blogged earlier this week about how we use marginalia to log the controller and action that each SQL query came from. We also tag each log line in production with a unique ID for each request and the account ID of the user. This has two big benefits: we can easily grep out one entire request with the unique ID, and we split logs based on the account ID to narrow down the search. Accomplishing this in Rails 3.2 is easy, thanks to ActiveSupport::TaggedLogging. Here’s a simple example of getting it running, crack open config/application.rb: YourApp::Application.configure do config.log_tags = [ :uuid ] end After bouncing your server, you’ll start to see a unique ID for each request in the log: [71ba53fd717c67a6677a058f4a5acdf4] Processing by ProjectsController#index as HTML The config.log_tags array can also take a static string, or a proc/lambda to allow for more customization. I’ve pulled out some more examples to check out as well. Console Mastery There’s a lot more to the WebKit console than console.log. I remember when Firebug utterly changed how I worked with JavaScript. After learning some tricks out of this codebase and some great blog posts lately, I’m starting to feel that way again. Two examples are console.warn(), and console.group() / console.groupEnd(). Warn adds the yellow icon, and grouping allows you to collapse/expand similar log messages: These are only enabled when in development mode, but it’s really simplified how we dealing with log messages and nailing down performance issues. Another great example is console.profile() and console.profileEnd(). There’s an entire tab for performance tuning and timing on the WebKit console. How did I miss this!? This tool has been really useful for simply visualizing what’s slow. It’s a bit obtuse at times, and the nesting of how functions are called/displayed is a little wacky, but definitely check it out. Finally, Chrome has a little piece of code that can show you how many milliseconds it’s been since the page load: chrome.csi().pageT // 2933006.221 This feels like it should be standardized somehow! In the meantime there’s a small snippet you can toss in to get it working in any browser. Overload! Hopefully you’ve picked up something new so far! There’s plenty more on the slides, along with some notes that conference-goers took. One huge take-away I’ve gotten from the Basecamp code base is that we’re still evolving and learning, and there’s no reason to stop!

We’re looking for one or two more folks to join our support team. Specifically, we’re after some great people in the UK time zone (or close to it) to cover business hours in that part of the world. Launching the all-new Basecamp got a lot of people excited and we want to expand our team to cover business hours in more time zones. You’ll be responsible for providing tremendous customer service via email for Basecamp, Basecamp Classic, Highrise, Backpack, and Campfire. You’ll also help us answer questions via Twitter, create and edit help documentation, and maybe run some online classes. You’ll be expected to answer about 75 emails per day once you’re fully up to speed (2-3 months on-ramp). This is a significant volume, so be sure that you’re ready and able to deal with that kind of daily load. We’re looking for some great writers who love helping our customers, so you should enjoy making complicated situations simple and painless and have a passion for our products. If you live in the UK (or thereabouts) and want to join Ann, Chase, Emily, Kristin, Merissa, Michael and me in making our customers happy, please apply! How to apply Please submit a cover letter explaining: Why you want to work in customer support. Why you want to work at 37signals and not somewhere else. A description of a great customer service/support experience you had recently, and what made it great. Also, attach the following writing samples: Explain in 3 paragraphs or less why a customer would pick Basecamp vs Highrise. Respond to a customer asking for project categories in Basecamp that it’s not something we offer, and suggest she work with the project titles and stars to add organization. A company using our job board failed to find to find a suitable candidate and wants a refund. Respond that we don’t offer refunds for job postings. We offer heaps of lovely benefits, plus a progressive work environment. Starting salary is $45k USD, depending on experience. Email everything to jointheteam@37signals.com. Include “Customer Support” in the subject line. If you’re attaching a resume, please send it as a PDF. Note: We look favorably on people who get creative with their applications. We look forward to hearing from you!

We just launched a brand new feature in the all new Basecamp that is handy in a bunch of different ways. Now you can move to-do lists, discussions, text documents, files, and calendar items between projects or into a brand new project. Moving items between projects is especially useful in a few specific cases (and probably many we’ve never thought of). For one, people put things in the wrong place from time to time. Now it’s no big deal – just move it to where it should be. And then there are other times when you want to break a bigger project into a couple smaller projects. Now you can take a list or a message or anything from one project and start an entirely new project from that item. It’s great. Here’s a video showing you how it works. We think you’re going to love it. Thanks for using Basecamp! Wait, you aren’t using Basecamp? What are you waiting for?.

One of the things we’ve added to our applications in the last few months is a little gem that (among other things) adds a comment to each MySQL query that is generated by one of our applications. Now, when we look at our Rails or slow query logs, our MySQL queries include the application, controller, and action that generated them: Account Load (0.3ms) SELECT `accounts`.* FROM `accounts` WHERE `accounts`.`queenbee_id` = 1234567890 LIMIT 1 /*application:BCX,controller:project_imports,action:show*/ When we’re trying to improve a slow query, or identify a customer problem, we never have to go digging to understand where the query came from—it’s just right there. This comes in handy in development, support, and operations – we used it during a pre-launch review of unindexed queries in the brand new Basecamp which launched a couple months ago. If you combine this with something like pt-query-digest, you end up with a powerful understanding of how each Rails action interacts with MySQL. It’s easy to add these comments to your Rails application in a relatively unintrusive way. We’ve released our approach that works in both Rails 2.3.x and Rails 3.x.x apps as a gem, marginalia. marginalia (mar-gi-na-lia, pl. noun) — marginal notes or embellishments Merriam-Webster We’ve been using this in production on all of our apps now since December, ranging from Rails 2.3.5 to Rails master and Ruby 1.8.7 to 1.9.3. You should be able to have it running in your application in a matter of minutes. It’s worth acknowledging that anytime you modify the internals of something outside your direct control there are risks, and that every function call adds some overhead. In our testing, these have both been well worth the tradeoff, but I absolutely encourage you to consider the tradeoff you’re making for yourself every time you instrument or log something. You may certainly have a different set of tradeoffs, and you should absolutely test on your own application. Have a suggested improvement to our sample code or another way to do this? We’d love to hear it. Thanks to Taylor for the original idea, and to Nick for helping to extract it into its own gem.

“Oh, that’s not my job,” is the sound of doom. Maybe not imminent doom, but doom indeed. It’s the magic inflection point when a company becomes too big (even if only psychologically) for any single employee to give a rat’s ass about job numero uno: Making shit work. No profession is immune. You can have designers who oh-thats-not-my-job to get the JavaScript they wrote to work, programmers who cry for operations to make their slow code run on time, operations people who refuse to answer customer complaints from their network outage, and on and on. Once the mentality cements, everything is eventually someone else’s job, and they’re being a toad for inconveniencing you with it. And besides, it’s easy to put it on somebody else, right? Everybody else’s job is easy! Departmental hedges grow fast and tall if not trimmed with vigor. It’s the natural path unless you take steps to fight it. That’s why, at 37signals, we all chip in when lots of customers have questions after a new product launch and customer support is overwhelmed. It’s why programmers will wake up in the middle of the night if a sql query tipped over and needs an urgent rewrite until faster servers can arrive. Don’t let your company culture become one where certain people are too good to do the jobs that need doing. Making shit work is everyone’s job.

My dad is a firefighter and fire captain in Niagara Falls, NY. When I told him I had on-call duty for my new job, he was beyond excited. After relaying stories of waking in the middle of the night to head into the hall and getting overtime from his buddies that didn’t want to wake up for work were sick, I had to explain that it’s a different type of action for me. I face issues of a less life-threatening variety, but there’s still plenty of virtual fires to extinguish. Here’s a primer for what being an on-call programmer means at 37signals. The routine On-call programmers are rotated around every 2 weeks, and all 8 programmers currently get their fair share of customer support and interaction. Usually we have 2 to 3 programmers on-call on a given week. Support issues from email and Twitter are routed through Assistly, and handled by our awesome support team. If there’s an issue they can’t figure out, we get a ping in our On Call room in Campfire to check out a certain issue: These issues range in severity, and they might look like: Got an “Oops” page (our standard 500 Server Error page) Can’t log in Page is messed up or broken Incoming email didn’t process or show up The next step is to FIX IT! If possible, get the customer’s problem figured out in the short term, and hopefully push out a bug fix to stomp it out permanently. Luckily, we’ve got a few tools to help us out debugging. We use a GitHub wiki to share code snippets that make some common issues go faster: This screenshot shows off a newer piece of our weapon rack as on-call programmers: 37. 37 is our internal tool for doing pretty much anything with our production servers and data. Some of what it covers: SSH into any box on our cluster Watch a live feed of exceptions streaming in Fire up a Rails or MySQL console for any application Grep or tail logs over multiple days from a specific account 37 has helped immensely by just the virtue of being a shared tool that we can use Ruby (and Bash!) to automate our daily work. It started as a small set of scripts tossed around on various machines, but it really became a vital part of our workflow once we made it into a repo and documented each command and script. Once the issue has been debugged/solved/deployed, we’ll log the issue as fixed on Assistly. The support crew handles telling the customer it’s fixed, but sometimes I’ll jump in if it’s a developer/API related issue. Other channels There’s a few other channels we pay attention to throughout the day as well. In Campfire we get alerts from Nagios about things that might be going wrong. Here’s one such example: a contact export from Highrise that was stuck in the queue for too long. Another incoming channel is the API support google group. Although this is more of a support forum for existing API consumers, we’ll try to jump in and help debug issues if there’s time left over from handling direct customer issues. Fire! Our on-call emergencies come in many different flavors. An all-out, 5 alarm fire is rare, but does happen. Typically it’s a slow burning fire: something is smoldering, and we need to put it out before it gets worse. We’ll learn of these problems in a few ways: Nagios alert in Campfire about downtime/inaccessibility Twitter eruption Support ticket flood Once we have a handle that it’s not a false alarm, we’ll update the status site and notify customers via @37signals. The number one priority from there is putting out the fire, and it usually involves discussing our attack plan and deploying the fix. If the fire is really getting out of control and tickets are piling up, sometimes we’ll help answer them to let them know we’re on it. The status site also knows everyone’s phone numbers, so if it’s off-hours we can text and/or call in backup to help solve the problem. Here’s a few tips I’ve learned from our fires so far: Be honest Given I’m relatively new to our apps and their infrastructure, I don’t know my way around 100% yet. Being honest about what you do and don’t know given the information at hand is extremely important. Be vocal about what you’re attempting, any commands you might be running, and what your thought process is. Jumping on Skype or voice iChat with others attempting to debug the problem might also be useful, but if someone is actively trying to hunt down the problem, be aware that might break their flow. Cool your head Each problem we get is different, and it’s most likely pretty hairy to solve. Becoming frustrated at it isn’t going to help! Stepping away might not be the best way to de-stress if the fire is burning, but staying calm and focused on the problem goes a long way. Frustration is immediately visible in communication, even via Campfire, and venting isn’t going to solve the problem any faster. Write up what happened Post-mortems are an awesome way to understand into how the fires were put out, and what we can do better the next time one happens. Our writeups get pretty in-depth, and knowing the approach used can definitely help out the next time. Especially since we’re remote and spread across many different timezones, reviewing a full report and reflection on the problem is extremely helpful, no matter how small the fire was. There is no undo! Issues that crop up repeatedly always have a larger fix in mind. Losing work due to browser crashes and closed windows was a large support issue, until autosave came to the rescue. When not putting out fires, on-call programmers are working on “slack” issues like these, or fixing other long-standing bugs. Experimentation with how on-call works is definitely encouraged to help our customers faster, and I’m sure our process will continue to evolve. It’s not perfect (what process is?), but I can definitely say it’s been improving. Putting out our virtual fires is still a lot of fun despite all of the stress!

It’s always satisfying to announce the release of one of the top Basecamp feature requests. Today is one of those days. We’re glad to announce that to-dos with due dates now show up on the all new Basecamp calendar. Prior to this update, only events that were directly added to the calendar showed up on the calendar. But as of today, any to-dos with due dates also show up on the calendar. And just like events, you can drag and drop to reschedule a to-do. Very handy. Now you have a much better view of everything that’s due in a project. To-dos, events, meetings, project phases, etc. – now you can see them all at the same time on a single calendar. We’re really happy with how it turned out. We hope you love it. Here’s a quick video showing you how it works.

The signs are all here: There’s now an incubator on every corner, even your uncle is donning angel wings, and IPO expectations for Facebook are exceeding the hype for a new Twilight movie. But that’s all circumstantial evidence. What we needed was some public testimony to really put everyone in the right frame of mind. Enter our trusted troubadour of bullshit, TechCrunch: My best guess is that it is about to get crazy. And, only fools sit on the sidelines. Many strong and older entrepreneurs that I know are wealthy today because they made intelligent decisions during the dot-com bubble of the late ’90s. Success was not easy then, and it will not be easy now, either. But, the likelihood of a great outcome is much higher in a boom. There are a lot of newly minted entrepreneurs that pursue their dream company in a halfhearted way. You may tinker with your idea while toiling at a day job. You may refuse to put in the work required to recruit the best talent. You might be afraid of launching an imperfect product. Or, you may put a mediocre effort into fundraising. Let me translate: Dude, you’re going to miss riding this bull onto the bubble if you do not get on it RIGHT. FUCKING. NOW! Didn’t you see that someone just made A BILLION DOLLARS? Why wasn’t that you, lazy schmuck? Don’t answer that—there’s no time to look at the past, just quit your job, and come out here to the Valley post-haste. Sand Hill road just scored a fresh load of loot. Now I get the basic psychology. Someone just won the billion dollar startup powerball and now everyone wants to make sure they bought a ticket for the next drawing. And why not? While the past season of tech IPOs has been full of duds, we still have the big baller Facebook coming up for a shot. And if red 32 hasn’t hit for the last few IPOs, it’s bound to do for this round. COME ON LUCKY 32!!! But let’s calm down. Sooner or later, the market is going to sort these things out, and all will be right as rain. That’s evident with the Groupon fiasco. They’ve restated their accounting numbers endlessly and the stock has finally tanked. At the end of the day, the rules of accounting will blow through all the smoke and the mirror will show a face with no make-up. Are you calm? Good. Now get ready to rage right back up. The new JOBS Act that was just passed with the help of a thousand VCs stomping it down the throat of Congress undoes all that. Matt Taibbi from Rolling Stones reports: Ostensibly, the law makes it easier for startup companies (particularly tech companies, whose lobbyists were a driving force behind passage of this law) attract capital by, among other things, exempting them from independent accounting requirements for up to five years after they first begin selling shares in the stock market. The law also rolls back rules designed to prevent bank analysts from talking up a stock just to win business, a practice that was so pervasive in the tech-boom years as to be almost industry standard. Now isn’t that swell. Enable people with an extreme financial incentive to spin the truth, or outright lie about the numbers, a 5-year get-out-of-jail cover, and what are you going to get? JOBS, silly! Duh! Buckle your seat belt, Dorothy, ‘cause Kansas is going bye bye.

When developers first discover the wonders of test-driven development, it’s like gaining entrance to a new and better world with less stress and insecurity. It truly is a wonderful experience well worth celebrating. But internalizing the benefits of testing is only the first step to enlightenment. Knowing what not to test is the harder part of the lesson. While as a beginner you shouldn’t worry much about what not to test on day one, you better start picking it up by day two. Humans are creatures of habit, and if you start forming bad habits of over-testing early on, it will be hard to shake later. And shake them you must. “But what’s the harm in over-testing, Phil, don’t you want your code to be safe? If we catch just one bug from entering production, isn’t it worth it?”. Fuck no it ain’t, and don’t call me Phil. This line of argument is how we got the TSA, and how they squandered billions fondling balls and confiscating nail clippers. Tests aren’t free (they cost a buck o’five) Every line of code you write has a cost. It takes time to write it, it takes time to update it, and it takes time to read and understand it. Thus it follows that the benefit derived must be greater than the cost to make it. In the case of over-testing, that’s by definition not the case. Think of it like this: What’s the cost to prevent a bug? If it takes you 1,000 lines of validation testing to catch the one time Bob accidentally removed the validates_presence_of :name declaration, was it worth it? Of course not (yes, yes, if you were working on an airport control system for launching rockets to Mars and the rockets would hit the White House if they weren’t scheduled with a name, you can test it—but you aren’t, so forget it). The problem with calling out over-testing is that it’s hard to boil down to a catchy phrase. There’s nothing succinct like test-first, red-green, or other sexy terms that helped propel test-driven development to its rightful place on the center stage. Testing just what’s useful takes nuance, experience, and dozens of fine-grained heuristics. Seven don’ts of testing But while all that nuance might have a place in a two-hour dinner conversation with enlightened participants, not so much in a blog post. So let me firebomb the debate with the following list of nuance-less opinions about testing your typical Rails application: Don’t aim for 100% coverage. Code-to-test ratios above 1:2 is a smell, above 1:3 is a stink. You’re probably doing it wrong if testing is taking more than 1/3 of your time. You’re definitely doing it wrong if it’s taking up more than half. Don’t test standard Active Record associations, validations, or scopes. Reserve integration testing for issues arising from the integration of separate elements (aka don’t integration test things that can be unit tested instead). Don’t use Cucumber unless you live in the magic kingdom of non-programmers-writing-tests (and send me a bottle of fairy dust if you’re there!) Don’t force yourself to test-first every controller, model, and view (my ratio is typically 20% test-first, 80% test-after). Given all the hundreds of books we’ve seen on how to get started on test-driven development, I wish there’d be just one or two that’d focus on how to tame the beast. There’s a lot of subtlety in figuring out what’s worth testing that’s lost when everyone is focusing on the same bowling or bacon examples of how to test. But first things first. We must collectively decide that the TSA-style of testing, the coverage theater of quality, is discredited before we can move forward. Very few applications operate at a level of criticality that warrant testing everything. In the wise words of Kent Beck, the man who deserves the most credit for popularizing test-driven development: I get paid for code that works, not for tests, so my philosophy is to test as little as possible to reach a given level of confidence (I suspect this level of confidence is high compared to industry standards, but that could just be hubris). If I don’t typically make a kind of mistake (like setting the wrong variables in a constructor), I don’t test for it.

In February I got a chance to chat with Paul Budnitz, founder of Budnitz Bicycles. “I design and create beautiful things and then create businesses to sell them,” Paul says. He’s been a professional photographer, shot films, owned a company modifying vintage clothing, turned making handmade microphones out of a garage into a multi-million dollar business — the list goes on. “I started out in eighth grade selling fireworks to all my friends in school and we actually programmed all our orders on the mainframe computers at the University of California, which at that time they were teletype computers. And we didn’t really know that those computers were controlled by the Department of Defense. So I eventually got arrested and suspended. That was my first business.” In 2002 Paul founded Kidrobot, which makes limited-edition art toys. “I like to make immaculate products,” Paul says, “and if I run the business myself I get to do it my way.” Kidrobot uses Basecamp to communicate with suppliers and distributers over four continents with hundreds of active projects. Budnitz Bicycles, Paul’s latest endeavor, uses the new Basecamp to work with manufacturers and suppliers around the globe. The bikes are gorgeous! Paul let me take one for a spin and it was honestly the most comfortable ride I’ve ever had. You can find out more about Budnitz Bicycles at budnitzbicycles.com and be sure to follow @budnitzbicycles on Twitter. Also, if you missed the profile we did on Happy Cog you can find it over here.

I’m starting to believe nothing should be designed in a day. Working a full-night’s sleep into your design process is as important as anything else you do. Morning tells the truth.

Designer Aaron Draplin lays down 50 points in 50 minutes the only way he knows how – bullshit free. Lots of great advice in here wrapped in great fun. Definitely worth your time.

Fantastic insight from Neil deGrasse Tyson about how important it is to be sensitive to someone’s current state of mind when you are trying to teach or persuade. You don’t teach with facts alone. You have to understand how those facts/thoughts are received by the person on the other end. And to do that, you have to understand what’s already in their head and how those ideas got there. Teaching is about bringing facts and external sensitivity together to have impact. This is powerful stuff and a great lesson for everyone.

When we were building the new Basecamp, we wanted the foundation to be built on clean, modern underpinnings to take advantage of all the new wonderful features of HTML5. That meant we have to drop support for older browsers, like IE8, that have little or poor support for the HTML5 technology we are using to make Basecamp awesome for everyone. But, have no fear! We realize that a lot of people are stuck with IE8 (sometimes even IE7 or, yikes, 6), so we made sure that Chrome Frame works with Basecamp. Chrome Frame is available for IE 6-9 on Windows machines and can usually be installed without admin access. If you’re stuck with an older version of Explorer, check out Chrome Frame and get yourself a Basecamp account.

One of the super-cool benefits of working at 37signals is a membership in a CSA (Community Supported Agriculture). We get fresh fruits and veggies at the farmer’s market or delivered to our doors, which encourages us and our families to cook and eat healthier — and the brain food helps us stay at the top of our mental game too! It also feels great to support local farmers. We’re spread out all over the place, so the contents of our CSA boxes reflect what’s local and in season where we live. Clockwise from top left: A recent delivery from Greenling in Austin, Texas; Javan’s Romanesco broccoli from Sunseed Farm in Ann Arbor, Michigan; Eron’s winter veggies from Coon Rock Farm in Hillsborough, North Carolina; and Ann’s fan dance with lettuce from Irv & Shelly’s Fresh Picks in Chicago. My favorite part of belonging to a CSA is the surprise that comes with opening a new box and seeing what’s new — it’s turned me into a more adventurous cook. A few of my CSA-based creations: grapefruit-avocado salad, sesame broccoli with peanut sauce, Swiss chard frittata and an avocado-orange smoothie. Soup is perfect for using up a bunch of perishables all at once! Shaun’s vegetarian chili, and Will’s wife’s veggie-and-ham soup Kristin made a pie with apples from her CSA last fall, and I made a sweet potato meringue pie with mine. Hey, it doesn’t have to be 100% healthy all the time, right? Michael, 37signals’ resident foodie and showoff, whips up amazing creations with his CSA goodies from Irv & Shelly — I’m considering moving to Chicago so he’ll feed me more often. Michael’s masterpieces, clockwise from top left: red quinoa with roasted cauliflower, dried cranberries, and toasted pine nuts; spinach tomato scramble with Sunday bacon; parsnip and potato latkes; mushroom ginger soup with hato mugi; squash and chickpea Moroccan stew with hand-rolled couscous; and poached eggs with sautéed spinach, toasted pecans, and parmesan. If you’re interested in CSAs in your area, check out localharvest.org. Bon appetit!