Vikinghammer 2013-09-24T07:14:39-07:00 Sean Schulte Mailgun and Go, go-mailgun 2013-09-24T02:00:00-07:00 <p>Lately, I've been playing with Go. The language itself is small enough that I quickly got to the point that I needed to write an app in order to feel like I was learning anything more. As it happens, earlier this year, I wrote an app called "MLB Notifier" that monitors the MLB datafeed for changes and emails me* about interesting changes. That was written in Java, and runs on Google App Engine. It seemed like a good opportunity to rewrite an app in Go; I'd get to deal with network communication, JSON parsing, sending email, and a not-insubstantial amount of logic.</p> <p><em>* Note to MLB's lawyers: it only sends emails to me, and nobody else. This is an individual, non-bulk, non-commercial use.</em></p> <p>This isn't about MLB Notifier. It's about email. From App Engine, I was able to just send messages from my Gmail account, because it's running on my Google account in Google infrastructure. For some reason, I assumed it wouldn't be an issue to just keep doing that from a VM running somewhere on the internet. But when I tried sending emails via SMTP using my Gmail credentials*, I immediately received an email from Google saying "Suspicious sign in prevented". Seems like someone trying to log into my Gmail account from a server in Indonesia seems weird to Google? Also, it turns out my VM lives in Indonesia.</p> <p><em>* Since this would require me to leave those credentials sitting on that VM, I already didn't like this solution.</em></p> <p>Enter Mailgun. My team uses it at work (in Ruby), and it seemed simple and effective. Googling around for how to use it from Go didn't surface any useful results (come on, you guys, I can't be the first person to do this), so I had to figure it out on my own. Mailgun provides solid API documentation, and I was able to convert their Ruby/PHP/Curl examples fairly easily.</p> <p>They have a very easy endpoint that you hit over HTTPS, which I personally think is nicer than having to actually deal with SMTP myself. The basic process for sending an email is:</p> <ul> <li>Send POST variables designating the sender, recipient, subject, and message body</li> <li>Set the Content-Type to application/x-www-form-urlencoded</li> <li>Set HTTP Basic Authentication to log in with api:[whatever Mailgun says is your API key]</li> <li>Send this request to the special endpoint that Mailgun gives you, which has your username in it</li> </ul> <p>You can see <a href="">all the code on GitHub</a>, in my new <a href="">go-mailgun</a> project.</p> <p>I wanted to pass a struct representing my message, which is really easy to define in Go:</p> <pre><code>type Message struct { FromName string FromAddress string ToAddress string Subject string Body string } </code></pre> <p>Along with a method to format the from name/address:</p> <pre><code>func (m Message) From() string { return fmt.Sprintf("%s &lt;%s&gt;", m.FromName, m.FromAddress) } </code></pre> <p>So then I just define my <code>Send</code> method:</p> <pre><code>func Send(message Message) error { client := &amp;http.Client{} </code></pre> <p>Here we set up the POST variables based on the Message provided:</p> <pre><code> values := make(url.Values) values.Set("from", message.From()) values.Set("to", message.ToAddress) values.Set("subject", message.Subject) values.Set("text", message.Body) </code></pre> <p>And then construct an HTTP request with the Content-Type header and Basic Auth:</p> <pre><code> request, _ := http.NewRequest("POST", ApiEndpoint, strings.NewReader(values.Encode())) request.Header.Set("content-type", "application/x-www-form-urlencoded") request.SetBasicAuth("api", ApiKey) </code></pre> <p>Then we send the request (note that Go lets us "defer" closing the body until we return from this method, which is <em>very</em> convenient, in case any of you have experience writing deeply nested try/catch/finally blocks in Java to do the same thing ... actually, I'm sorry I mentioned it, let's just move on):</p> <pre><code> response, e1 := client.Do(request) if e1 != nil { fmt.Println("Failed to send request") fmt.Println(e1) return e1 } defer response.Body.Close() </code></pre> <p>And I read the response here, even though I really don't need it. Although printing it in the logs was useful, at the beginning when I hadn't set the Content-Type and Mailgun's error message told me it couldn't send a message without a "from" parameter. I took that to mean that it wasn't receiving the parameters, and it wasn't long before I figured out it needed the Content-Type.</p> <pre><code> body, e2 := ioutil.ReadAll(response.Body) if e2 != nil { fmt.Println("Failed to read response") fmt.Println(e2) return e2 } fmt.Println(string(body)) return nil } </code></pre> <p>So that's easy! But that's if you want to write the method yourself, which I don't intend to do again. Using it is even easier, as demonstrated in this degenerate example app:</p> <pre><code>import ( "" ) func main() { mailgun.ApiEndpoint = "" mailgun.ApiKey = "YOURKEY" go func() { err := mailgun.Send(mailgun.Message{ FromName: "Foo Bar", FromAddress: "foo@bar.test", ToAddress: "recipient@bar.test", Subject: "This is an example message", Body: "It's pretty easy to send messages via Mailgun!", }) if err != nil { // you can handle sending errors here } }() } </code></pre> <p>I set my endpoint/key once (in my app I do it in <code>main.main</code>, having read the values from a YAML file), and then send the email within a goroutine. In this case it's an anonymous function, but obviously it doesn't have to be; in my MLB Notifier application I name a separate method that calls <code>mailgun.Send</code>, because in that case it makes the code clearer.</p> <p>So, I was pleased by how simple it was, and I have been pleased with Mailgun's performance delivering the emails. Notably, except for one evening where I was seeing weird behavior:</p> <ul> <li>(21:42) SD tied it up in the 9th, 2-2</li> <li>(21:42) SD broke the tie in the 9th, 3-2</li> <li>(22:42) SD tied it up in the 9th, 2-2</li> <li>(22:59) SD broke the tie in the 9th, 3-2</li> </ul> <p>That 9th inning took over an hour, and the same team tied it up and took the lead multiple times? I investigated my logic repeatedly, and for the life of me I could not figure out how this could be happening.</p> <p>And then, I received an email from Mailgun with a link to a <a href="">post explaining what happened</a>. I was satisfied with the explanation and was impressed and pleased that they got right out in front of it without any denials or excuses. So I'll continue to happily use Mailgun to deliver interesting events in baseball games to myself ... unfortunately for them I don't really have to think about it any more, now that it's running smoothly.</p> Rails has_secure_password, a reminder 2013-02-20T01:00:00-08:00 <p>I'm using the new <code>has_secure_password</code> feature in Rails (I believe it was introduced in Rails 3.1) for user authentication. It uses bcrypt under the hood, which I'd used directly in the past. But <code>has_secure_password</code> seems to be the cool new way to do this, and it does seem pretty easy.</p> <p><code>has_secure_password</code> gives you a bunch of things for free: password hashing and salting, authenticating against the hashed password, and password confirmation validation.</p> <p>But all the tutorials and explanations I've seen omit a few key details in the name of simplicity. Namely, they don't show you what you need to do for the password confirmation to work, and they don't show you how to validate the password correctly when the User object is updated (ie, they only show you what to do when it's created).</p> <p>The <a href="">RailsCast</a> on the topic makes it seem <em>sooooo</em> easy, which is I guess the point, but if you just do what they do you'll end up confused and with something that doesn't work.</p> <p>Their User model looks like this:</p> <pre><code>class User &lt; ActiveRecord::Base attr_accessible :email, :password, :password_confirmation has_secure_password validates_presence_of :password, :on =&gt; :create end </code></pre> <p>Note that it only validates the password when the User is created; that's because they want you to be able to change the username later, without having to re-enter the password. But if you add a length restriction to your password (which you should), then later on when the user changes it, the minimum length won't be validated.</p> <p>Additionally, and I think this is key, the RailsCast shows you the User model, and how to authenticate, but never actually shows how to create a user and have the password_confirmation validated.</p> <pre><code>user = =&gt; "") user.password = "password" user.valid? =&gt; true </code></pre> <p>I would've thought that would fail validation, because there's no password_confirmation set. But this is the key, the thing that apparently isn't mentioned anywhere: <strong>the password confirmation is only validated if you attempt to set it</strong>.</p> <pre><code>user = =&gt; "") user.password = "password" user.password_confirmation = "otherpass" user.valid? =&gt; false user.password_confirmation = "password" user.valid? =&gt; true </code></pre> <p>Okay, so you have to actually set the password_confirmation or else it won't do anything. But how do you make your validation work when you're updating the user?</p> <p>Here's a very basic User model that will do that:</p> <pre><code>class User &lt; ActiveRecord::Base attr_accessible :id, :username, :password, :password_confirmation has_secure_password validates :username, :presence =&gt; true, :length =&gt; { :minimum =&gt; 3 }, :uniqueness =&gt; true validates :password, :length =&gt; { :minimum =&gt; 8, :if =&gt; :validate_password? }, :confirmation =&gt; { :if =&gt; :validate_password? } private def validate_password? password.present? || password_confirmation.present? end end </code></pre> <p>Here, it will validate the password and its confirmation <a href="">if either the password or the confirmation are set</a>, or if it's a new User with no password set. This will allow you to update the username without setting the password, but will actually validate the password if you attempt to set it. (And still has the restriction that it only attempts to validate the confirmation if you set password_confirmation.)</p> AppEngine HTTP Headers 2013-01-29T03:00:00-08:00 <p>So, I'm working on a new Android app* which requires user authentication and storing data to a backend service. Since pretty much everyone with an Android phone has a Google account, and Google is doing a good job of implementing OAuth 2 for me, I figured I'd just use their authentication. <a href="">Tim Bray alerted me</a> to <a href="">the cool new way of authenticating</a> which doesn't require the user to enter any passwords, and may not even require the user to even click anything.</p> <p><em>* It's a game, with asynchronous multiplayer functionality.</em></p> <p>After being confused for a while about how complicated it was supposed to be -- I thought you have to pass in the token from <code>GoogleAuthUtil.getToken</code> into something else to complete the authentication and pass a cookie back to AppEngine to maintain an HTTP session, when in fact you can just pass the token directly to your service to verify identity -- I was ready to start moving forward.</p> <p>I decided that I'd pass the token as a custom HTTP header, so I don't have to either put it in the query parameters (which is dangerous even on HTTPS) or pollute the JSON I'm passing to the service. I figured <code>X-GoogleAuthToken</code> would be a good option for a header name. And this is where our story really begins.</p> <p>On the local dev service, <code>X-GoogleAuthToken</code> worked fine, and I was able to authenticate using the token. But when I deployed to production, the value for the header always came through as null.</p> <p>So I wrote an extremely simple servlet to test out the headers.</p> <pre><code>public void doGet(HttpServletRequest req, HttpServletResponse resp) throws IOException { resp.setContentType("text/plain"); Enumeration headers = req.getHeaderNames(); while (headers.hasMoreElements()) { String name = (String)headers.nextElement(); String value = req.getHeader(name); resp.getWriter().println(String.format("%s: %s", name, value)); } } </code></pre> <p>I passed in the following headers*:</p> <pre><code>GET /lettergarden HTTP/1.1 X-Googleauthtoken: blah blah Googleauthtoken: test Fakeheader: here X-Gauthtoken: howdy X-Google-Auth-Token: 123 X-G-Auth-Token: boosh </code></pre> <p><em>* It was supposed to be "X-GoogleAuthToken" and "fakeheader" and "X-Gauthtoken", but I'm using an app called "HTTP Client" that apparently, and I just learned this, title cases the headers before sending them. Seems wrong?</em></p> <p>And on the local server, here's what I got:</p> <pre><code>Host: localhost:8888 User-Agent: HTTP%20Client/0.9.1 CFNetwork/454.12.4 Darwin/10.8.0 (i386) (MacBookPro5%2C4) X-Googleauthtoken: blah blah Googleauthtoken: test Fakeheader: here X-Gauthtoken: howdy X-Google-Auth-Token: 123 X-G-Auth-Token: boosh </code></pre> <p>Okay, that's everything I passed in, plus the Host and User-Agent. Nothing to suggest there are dragons approaching.</p> <p>Here's what I get after deploying to production:</p> <pre><code>Host: some-project-this-isn' Fakeheader: here Googleauthtoken: test X-Gauthtoken: howdy X-G-Auth-Token: boosh User-Agent: HTTP%20Client/0.9.1 CFNetwork/454.12.4 Darwin/10.8.0 (i386) (MacBookPro5%2C4) X-AppEngine-Country: US X-AppEngine-Region: il X-AppEngine-City: chicago X-AppEngine-CityLatLong: 41.878114,-87.629798 </code></pre> <p>The <a href="">AppEngine documentation says</a> they'll add X-AppEngine-Country, X-AppEngine-Region, X-AppEngine-City, and X-AppEngine-CityLatLong as a service to you so you can do some location stuff easily. They also say they <em>will</em> remove some headers from requests: Accept-Encoding, Connection, Keep-Alive, Proxy-Authorization, TE, Trailer, Transfer-Encoding.</p> <p>You may have noticed, however, that it <em>also</em> stripped X-Googleauthtoken and X-Google-Auth-Token from the request. The documentation does not indicate anywhere that AppEngine will strip any HTTP header that starts with "X-Google". And it <strong>does not</strong> strip headers that start with "Google" (which obviously you shouldn't do).</p> <p>I'd rather Google didn't bogart the entire X-Google* namespace if they're not even using any part of it. But, frankly, it wouldn't be a big deal at all if they'd just <em>document that they're doing it</em> and <em>make the dev environment mirror the actual production environment</em>.</p> <p>I spent a few hours combing Google search results, AppEngine documentation, and StackOverflow for why this was happening. Maybe, having written this, the next poor sap to be bitten by this won't have to waste so much time.</p> Dependo Pull Request 2013-01-29T02:00:00-08:00 <p>Last night, I got my first pull request on GitHub from someone I don't know. Pretty cool.</p> <p>As you may know, <a href="/2012/01/04/ruby-dependency-injection-introducing-dependo/">Dependo is a very simple dependency injection library for Ruby</a>, which just overrides the <code>#method_missing</code> method and lets you inject methods onto an object just by mixing in the Dependo module.</p> <p>Well, apparently someone wanted to be able to see if methods have been injected onto the object. I'd been doing that by calling <code>Dependo::Registry#has_key?</code>, but this new way is better.</p> <pre><code>module Mixin def respond_to?(key, include_private=false) if Dependo::Registry.has_key?(key) true else super(key, include_private) end end end </code></pre> <p>So I quickly accepted the pull request. I probably wouldn't have thought of this without some guy finding my library, using it, and contributing the idea and making my project better.</p> <p>In case you needed another reason to open source your stuff, this is a great example of why it's a good thing.</p> <p>Thanks, <a href="">Tomas</a>!</p> Whither Scoreboard? 2012-12-27T00:00:00-08:00 <p>My most popular apps have been MLB Scoreboard (121,664 users) and NFL Scoreboard (61,994 users). In both cases, the apps scratched an itch, started small, and grew from there.</p> <h1>The birth of MLB Scoreboard</h1> <p>I started MLB Scoreboard because MLB At Bat for Android was <em>awful</em> two years ago.* I needed an app that would tell me the scores of baseball games, with as few clicks as possible, with as many games on the screen as possible, and without taking several extra seconds to load for no reason.</p> <p><em>* Now, it's merely inadequate. Big step up?</em></p> <p>So, MLB Scoreboard was very, very simple when it first started. At the beginning, it was just a scoreboard: one screen, which showed all the games for the day in the order they were played, with the linescore like you'd see at the stadium. There was also the ability to go forward and backward by a day at a time, to see the previous days' results or tomorrow's schedule. That was it.</p> <p>The app grew slowly. It took weeks to get to 100 users, and I was really excited. I kept adding new features: your favorite team would always appear at the top of the screen; I showed the current outs and runners on base; I let you click into a game to view the play-by-play and the hitting/pitching boxscore; I added highlight videos for each game; soon, there was a widget that showed today's game for your favorite team, so you didn't even have to wait for the app to open to get the score.</p> <p>By the middle of the summer in 2011, it had somehow shot past 1000 users. That was the number I'd barely dared to hope it'd get to. When people had asked when I was going to put ads in it, I'd been saying that I'd bother with that when it got to 1000; but by the time it happened, I was too concerned by the fact that MLB's license for their data says it's for non-commercial uses only. If I tried to make any money, it wouldn't be non-commercial any more. So I left the ads out.</p> <p>And it kept getting more popular. Near playoff time, people seemed to discover it. By the end of October, 7000 people were using it multiple times per day. On the modern internet, where ten million is the new one million, a few thousand is nothing. But for me, it was exciting.</p> <p>And so, with the baseball season winding down, it was time for football season to start. And it just so happens that the NFL's Android app is much, much worse than MLB's. Like, unforgiveably awful. I needed a better app to see the scores, if I was going to follow football.</p> <h1>The birth of NFL Scoreboard</h1> <p>NFL Scoreboard started the same way as MLB Scoreboard, and perhaps even more barebones: just a list of the games and their scores.</p> <p>And it similarly grew slowly, but not quite as slowly. Maybe the kind-of-popularity of MLB Scoreboard helped? It quickly shot past a few hundred users. By the end of the (much shorter) season, it had 1000 users.</p> <p>Despite being monumentally ugly, it had a few advantages over NFL's official app. The official app took 30 seconds to start, and after a couple of clicks you actually got to the scores, and you could see two games on a screen. My app started in under one second, and went straight to the list of scores, and you could see seven games on a screen.</p> <h1>A new baseball season</h1> <p>In March 2012, I was contacted by AT&amp;T, who wanted to promote MLB Scoreboard as a featured part of their new Android app store. Since baseball season was about to start, they wanted an entire baseball section. So they had me fill out some informational forms about the app, with logos and a description, etc. Pretty much all the same stuff you'd get from the actual Google Play (nee Android Market). I knew it wasn't going to be a big deal, but I told people that I was letting myself become foolishly excited for all the downloads I was going to get from AT&amp;T users.</p> <p>I should have known it wouldn't go that way. AT&amp;T, being AT&amp;T, never got their act together with the store. There was no baseball section. There was a sports section, but MLB Scoreboard was never in it. The only apps in it were those ultra-lame cookie-cutter ringtone/wallpaper apps. And every image on their app store site was broken. It was almost as poorly executed as AT&amp;T's actual phone service.</p> <p>But the 2012 baseball season rolled on anyway, and even without AT&amp;T's help, MLB Scoreboard had somehow gotten popular.</p> <p>At the start of spring training, there were 14000 installs (5000 active users). The rate of installation increased throughout March, and at the start of the season there were 26000 installs (14000 active users). Throughout the entire month, I avidly checked my download stats every morning, shocked that another few hundred people had downloaded it the previous day.</p> <p>Because there was suddenly so much interest, I felt I needed to improve the app. It was pretty ugly. So, I spent a few weeks rebuilding the entire UI, making it more modern and fitting better into the rest of Android (ie, ActionBar), and adding animations to make things look cooler and smoother. At first, people very vocally <em>hated</em> what I was doing, and I got a bunch of angry emails.</p> <p>And then, on Opening Day, the actual baseball-loving public started caring about baseball. And I got 1000 downloads that day. And the next. And more the next. For the first couple of weeks in April, I was getting 3000-4000 downloads every day. It slowed back down in May, to the point where I was only getting about 400-500 per day* for the rest of the season.</p> <p><em>* It's amazing how quickly I got used to that. A year earlier, I would have been amazing by having a total of 400 users.</em></p> <p>The growth rate stopped at the end of the playoffs, once again, peaking around 120K (63K active). Turns out people don't really care about the score of today's baseball game when there are no games.</p> <h1>A new football season</h1> <p>Right before the start of the 2012 football season, I received an email from a shady Russian company, offering to buy NFL Scoreboard from me for $1000. It was ugly, and only had 1000 users. I briefly considered it, but decided I'd rather make the app better, than sell out for what is basically a pittance.</p> <p>I modernized the app's interface, again with ActionBar. I added features, like full game stats, full play by play, and all the passing/rushing/receiving stats for the players. People seemed to love these new features. Every Sunday morning before the games, I got several thousand new downloads.</p> <p>After week two, NFL Scoreboard had shot up to 60000 users, and my growth rate was increasing.</p> <p>I was loving it.</p> <h1>First tangle with a legal department</h1> <p>On September 14, 2012, I received an email from Google. They informed me that NFL Scoreboard had been suspended, and removed from Google Play, for "alleged trademark infringement".</p> <p>They informed me that repeated violations of this nature would result in my entire developer account being suspended; this made me immediately fearful about MLB Scoreboard. I did not want to lose my developer account. Especially since my experience with Google is that they don't communicate with you when you try to appeal these decisions; you're just screwed.</p> <p>I contacted the NFL legal department. The email from Google had included a list of a few hundred other apps that had been caught for infringement; mine stood out like a sore thumb in that list, since every single other app that was caught was a lame, obviously infringing app called something like "New England Patriots Ringtone" or "Super Bowl Wallpaper" that had no features, and was probably just malware.</p> <p>In my email to the NFL, I asked if there were any changes they'd like to see to the app in order to get it reinstated. I told them I was willing to work together to find common ground; I was willing to remove features, to include a banner ad advertising their official app, et cetera.</p> <p>I pointed out that there was a small group of devoted users who loved the app, and that that implied there was room in the market for a lighter-weight option than the official NFL '12 app, and that was the role I was trying to fill.</p> <p>The NFL never responded. They had gotten what they wanted -- NFL Scoreboard was dead.</p> <h1>It happens again</h1> <p>I should have known I was flying too close to the sun, when I saw that Google Play was running its algorithmic app-finding helper thing, with the line "Users who downloaded MLB At Bat also downloaded MLB Scoreboard!" MLB would <em>have</em> to notice that, right?</p> <p>And a little bit after the end of the baseball season, I received an email from MLB.</p> <blockquote><p>Please read the attached letter from the legal department at MLB Advanced Media regarding your MLB Scoreboard mobile application.</p></blockquote> <p>Uh oh.</p> <blockquote><p>We write with respect to your unauthorized commercial use of MLB Material in your MLB Scoreboard application available in the Google Play market</p></blockquote> <p><em>Unauthorized commercial use</em>, did you say? I asked for an explanation, since I very explicitly had not attempted to gain any commercial benefit from this free app with no ads in it.</p> <blockquote><p>Even though the application is being offered for free and doesn’t contain advertisements, it’s still being made available in commerce, which in this case constitutes commercial use in violation of the copyright notice and license described in our letter. Since you do not have a license from MLBAM, you are not authorized to use our Materials in this way and we must again insist that you immediately remove this application from the Google Play Market, sign the letter in the spaces provided and return it to us</p></blockquote> <p>So, I wondered, just being in Google Play constitutes "in commerce"? I asked that.</p> <blockquote><p>Yes it does, as it would if the application were available in the iTunes App Store, the Windows Marketplace, or any other similar application distribution platform.</p></blockquote> <p>Since it was apparent that if any third party might potentially derive some indirect benefit from the existence of the app meant that the app was "commercial", I asked for an example of a non-commercial use. You know, since it didn't seem like there was much possibility of avoiding that.</p> <blockquote><p>If, moving forward, you have ideas for ways in which to use the Materials that you believe fall under the category of “individual, non-commercial, non-bulk use,” you can send them along to me and I will be able to confirm whether or not they do.</p></blockquote> <p>Seriously. That was MLB's response to "can you give me an example of a non-commercial use". I don't consider that an <em>example</em>.</p> <p>In the end, MLB forced me to remove MLB Scoreboard from the market. By my own hand. I didn't want to press them too hard, because they could just go to Google alleging copyright infringement, and I didn't want that to happen. Frankly, I'm glad MLB dealt directly with me. It was just painful, that I was forced to murder my own app.</p> <h1>End of an era?</h1> <p>So, my two best and most popular apps have been killed.</p> <p>Some people surely feel this is a good thing. Before they were removed, I was talking to someone from Google and when he heard what the apps were, he responded "so, they're just pure copyright infringement." Those poor little professional sports leagues need to be protected from thieving bullies like me, standing on their backs to hog all the money and glory for myself. Removing any and all competition from the market is how American capitalism is supposed to work, right?</p> <p>Others have pointed out that their vociferousness doesn't really make any sense. There's no conceivable way in which someone would download my apps so they could <em>stop</em> being a fan of the MLB or the NFL. I'm providing free labor to multiple-billion-dollar organizations in service of furthering the fanaticism of the people whose fanaticism makes them all their money.*</p> <p><em>* I wish I were slightly less of a baseball fan, or that I lived in the same market as my favorite baseball team. Then, I'd be more likely to cancel my package in response to MLB's hit job on me. It'd actually cost them money, for no benefit to them. But my money will continue to flow to them, because I lack the courage of my convictions. And I love baseball.</em></p> <p>Over the past two years, I've received requests from hundreds of people -- friends and strangers alike -- to make a version of my Scoreboard apps for their favorite sports. NBA, NHL, soccer. That will not be happening now. The poor little sports leagues can keep all the control they so crave, and they can continue to utterly fail to serve their customers.</p> <p>Unless one of the leagues comes to me and wants me to help them make an app that doesn't suck, and that people actually <em>want</em>, I'm done with sports apps.</p> Running two pieces of middleware with Dependo 2012-07-31T00:00:00-07:00 <p>Middleware is an awesome feature of Rack -- I can add functionality that wraps around an HTTP call without having to change the actual server.</p> <p>We use it with some success in our <a href="">r509-ca-http</a> project. r509-ca-http is a Certificate Authority based on <a href="">r509</a> that serves over an HTTP REST API. Its functionality is intentionally as simple as possible -- the r509-ca-http project is intentionally not responsible for storing information about certificates' validity or a record of which certificates have been issued. Its only responsibility is issuing and revoking certificates.</p> <p>But if we're going to actually run a CA, we need it to store validity information, among other things. Enter middleware.</p> <p>We created a new project, <a href="">r509-middleware-validity</a>, responsible for storing validity information (issuance and revocation) about every certificate into a Redis database. Originally, we structured it such that it relied on the server it was wrapping around to have a <code>#log</code> method on it:</p> <pre><code>module R509 module Middleware class Validity def initialize(app) @app = app end def call(env) status, headers, response = # this will just intercept the calls to /1/certificate/issue and ignore anything else if not (env["PATH_INFO"] =~ /^\/1\/certificate\/issue\/?$/).nil? and status == 200 "I intercepted /1/certificate/issue" end [status, headers, response] end end end end </code></pre> <p>In your <code></code>, you activate the middleware like so:</p> <pre><code>use R509::Middleware::Validity server = R509::CertificateAuthority::Http::Server run server </code></pre> <p>And on each call to the server, the middleware will be executed first; you need to send the call along to the server with that <code></code> line, and return the results after you're done. But because we're using that <code></code> method call, we're relying on our Sinatra server having a <code>#log</code> method; since it uses the <code>Dependo::Mixin</code>, and we added a <code>Logger</code> to the <code>Dependo::Registry</code> in our <code></code>, that method will be available.</p> <p>Which is pretty cool, and was working -- until we needed to add another piece of middleware. In addition to storing validity information in a database, we <em>also</em> need to store every issued certificate on disk. This is another thing we don't want to add to the HTTP service, so middleware is the perfect place for it. So we created <a href="">r509-middleware-certwriter</a> and got it saving every certificate that we issued. It was working great in testing ... and then we tried running the server with <em>both middlewares active at once</em>.</p> <pre><code>use R509::Middleware::Certwriter use R509::Middleware::Validity server = R509::CertificateAuthority::Http::Server run server </code></pre> <p>And as soon as we tried to issue a certificate, we got this:</p> <pre><code>I, [2012-07-31T14:42:15.260855 #9007] INFO -- : Writing serial: 1233561620675808731887525784788727751733782628003, Issuer: /C=US/ST=Illinois/L=Chicago/O=Ruby CA Project/CN=Test CA NoMethodError: undefined method `log' for #&lt;R509::Middleware::Validity:0x00000100c0dab8&gt; /Users/sschulte/code/r509-middleware-certwriter/lib/r509/middleware/certwriter.rb:35:in `rescue in call' /Users/sschulte/code/r509-middleware-certwriter/lib/r509/middleware/certwriter.rb:28:in `call' /Users/sschulte/.rvm/gems/ruby-1.9.3-p0/gems/rack-1.4.1/lib/rack/lint.rb:48:in `_call' /Users/sschulte/.rvm/gems/ruby-1.9.3-p0/gems/rack-1.4.1/lib/rack/lint.rb:36:in `call' /Users/sschulte/.rvm/gems/ruby-1.9.3-p0/gems/rack-1.4.1/lib/rack/showexceptions.rb:24:in `call' /Users/sschulte/.rvm/gems/ruby-1.9.3-p0/gems/rack-1.4.1/lib/rack/commonlogger.rb:20:in `call' /Users/sschulte/.rvm/gems/ruby-1.9.3-p0/gems/rack-1.4.1/lib/rack/chunked.rb:43:in `call' /Users/sschulte/.rvm/gems/ruby-1.9.3-p0/gems/rack-1.4.1/lib/rack/content_length.rb:14:in `call' /Users/sschulte/.rvm/gems/ruby-1.9.3-p0/gems/rack-1.4.1/lib/rack/handler/webrick.rb:59:in `service' </code></pre> <p>Turns out that when you're using two middlewares, the second one wraps around the server, and the first one wraps around the second one -- ie, they don't both somehow wrap directly around the server. So in our case, the <code>#log</code> method exists on the HTTP server so the validity middleware can use it, but it doesn't exist on the validity middleware so the certwriter middleware <em>can't use it</em> and dies.</p> <p>Here comes <a href="">dependo</a> to the rescue!</p> <pre><code>module R509 module Middleware class Validity include Dependo::Mixin def initialize(app) @app = app end def call(env) status, headers, response = "Now I can just use the log method directly" [status, headers, response] end end end end </code></pre> <p>Since the server is using <code>Dependo::Mixin</code>, it has a <code>#log</code> method. If we add <code>Dependo::Mixin</code> to both our middlewares, they each have a <code>#log</code> method and can wrap around each other in either direction.</p> <p>Maybe this is obvious, maybe pointless ... but I hadn't thought of it until I ran into it. This is how I got past it.</p> Using ActionBar 2012-07-01T00:00:00-07:00 <p>With MLB Scoreboard 2.0, I tried to "modernize" the interface with a slide-out menu. It didn't work, but today I'm trying to modernize it yet again. This time, I'm going with <strong>ActionBar</strong> as my attempt at making the interface newer and fancier; this is the direction Google wants you to go with your apps, many other app makers are doing it, and it unifies how the menu works between phones and tablets.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-07-01-14-28-27.png" title="New schedule" alt="New schedule" /></p> <p>You can see that the old bar with the arrows (to go forward or backward one day at a time) has been replaced with a blue ActionBar at the top of the screen.</p> <p>You can refresh the current data, or go forward or backward one day at a time with the buttons on the ActionBar, or use the "overflow" menu to access the rest of the features. If your phone has a physical menu button, that overflow menu icon doesn't appear; instead, the menu appears when you click your menu button.</p> <p>Even though ActionBar was introduced with Android 3.0, and changed further with 4.0, Google <em>does</em> provide a compatibility library that allows you to use it on previous versions of Android. I finally figured out how to get that to work, so MLB Scoreboard <em>still works</em> on pre-Honeycomb devices.</p> <p>This resolves the <a href="">menu junkiness</a> I mentioned in my last post, so I think people will like it more.</p> <p>You'll also note that the "R H E" has been moved to the top of the screen and only appears once, rather than in every single row. That saves a ton of wasted space. And the pitcher that got the save has been added back to the schedule, for those who care about that.</p> <p>If this sounds like it's moving in the right direction, <a href="">go download MLB Scoreboard 2.3 now</a>!</p> Information Density 2012-06-27T00:00:00-07:00 <p>I'm trying to sift through the complaints about the new MLB Scoreboard interface, to figure out which are knee jerk responses that don't need to be addressed, and which are on the money.</p> <p>I tried to chase the idea of a "modern" design, by aping the slide-out menu that a lot of other apps have adopted; part of that was that the menu doesn't cover the entire screen. In other apps that seems to work better, and I think the reason is that people don't want to sit and stare at the menu, they want to use it to get where they're going.</p> <p>But in MLB Scoreboard, one of the things people want to do is look at the schedule. And they have certain expectations about what that schedule is going to show them.</p> <p>I think that was one of the big, actual problems with the new layout. The original schedule contained the full linescore of every game, the starting pitchers, the winning and losing and saving pitchers, the current inning and bases and outs situation, and the latest play.</p> <p>But the new one only had the teams, the score, and the inning/base/out situation. It seemed like a big step backwards, because for what people were using the app for, it was.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-26-22-23-55.png" title="New slide-out menu" alt="New slide-out menu" /></p> <p>I've added more of the linescore, including not only runs, but also hits and errors. And the latest play is there for the games that are in progress; before games start, the starting pitchers are there, and after games are over the winning and losing pitchers are displayed.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-27-11-34-41.png" title="New slide-out menu" alt="New slide-out menu" /></p> <p>This is moving in the right direction, I think. Information density on the schedule screen has shot back up in the latest version. It still doesn't show the full inning-by-inning linescore, which I know will disappoint some people. But I continue to think I'm right that it's time to move away from that; it takes up <em>way</em> too much space, and it didn't look very good. It used to be able to show 3-4 games on one screen, now it can show 8.</p> <p>The schedule still slides out, but it now fills the whole screen. Again, having part of the screen essentially "wasted" was something that people didn't like, and I think it had to do with the information density issue. <em>Why are you leaving that space untouched and unused, and showing me so much less of the information I care about?!</em></p> <p>Oh, and one last thing. As part of my update, I jumped the <strong>targetSDKVersion</strong> up to the latest. That improves things, mostly, but it creates one huge problem. The menu button, where you can get to your settings and standings and such, <a href="">only appears on-screen on phones in ICS, not on tablets</a>. I mean, unless you use <strong>ActionBar</strong>, which of course is only available on Honeycomb and ICS ... which means 95% of my users can't use it.</p> <p>When people complain about fragmentation, this is pretty much what they're talking about. So I had to do something kind of stupid, which you probably saw in that screenshot. There's now a menu button on the screen, within my app. I don't use the actual device menu button any more, which is probably going to be fine going forward since the menu button is going away. But I know it sucks for tens of thousands of my users* who have an older phone that has a menu button. I know there's a better way to do this, one that isn't quite so lame. I'll figure out what that is soon, I'm guessing.</p> <p><em>* Myself included.</em></p> <p>Did I get it right? To find out, <a href="">go download MLB Scoreboard 2.2 now</a>!</p> Hating the new 2012-06-26T00:00:00-07:00 <p>Wow.</p> <p>I'm not really accustomed to being attacked like this by a group of people -- usually it happens face to face, one person at a time. But now, after the 2.0 update to MLB Scoreboard, there's been a flood of vitriol aimed squarely at me and my (rudimentary) design sensibilities.</p> <p>I thought the old design was getting old and crufty, and was kind of ugly. So I tried to fancy it up a bit. And I thought that was mostly successful, and I like the app more now.</p> <p>And then ... the users got their hands on it.</p> <p>Mike (Galaxy Nexus):</p> <blockquote><p>Backward step Big step backward. Settings menu disappeared. Slide out menu flat against background with no transparency: visually confusing. Push button menus. Aging icons. White background saps battery on AMOLED devices; no way to switch.</p></blockquote> <p>Ed (Galaxy SII):</p> <blockquote><p>"Improvement????" Now you get the scores on half of a screen? So pathetic I wish I could give zero stars. Actively looking for an alternative!</p></blockquote> <p>Tom (Galaxy Nexus):</p> <blockquote><p>Update This app sucks now!!!! Wtf!!! Half screen scores and now the team selection only shows home and away. Get with it. Rate 1 star. Don't download til its fixed. Lame.</p></blockquote> <p>Christine (Droid 4):</p> <blockquote><p>Update is terrible! This new update is aweful! Terrible! Put it back they way it was!!!</p></blockquote> <p>Mike (Galaxy S):</p> <blockquote><p>Alrite New update is bad but it doesnt freeze anymore put back scores the way they were please and will rate 5 stars</p></blockquote> <p>John (Acer Picasso):</p> <blockquote><p>Love the data provided but not the new layout I have to agree with the rest of the reviewers who want you to switch it back. the new layout isn't too bad on my tablet but terrible on my phone .</p></blockquote> <p>John (Galaxy Tab 10.1):</p> <blockquote><p>Update Latest update not as good as prior. Please revert.</p></blockquote> <p>B (Asus Transformer Pad TF300T):</p> <blockquote><p>Latest update seems to have removed the menu button which means I can't access current standings. Please restore that functionality.*</p></blockquote> <p><em>* I'm sorry to say, I'm not 100% sure I know what B is talking about here. The menu button still works on my phone ... maybe it doesn't on a tablet? Can anyone verify that for me?</em></p> <p>Vicki (Galaxy S):</p> <blockquote><p>Bring back the old version I can barely read the scores with it being condensed to one side. I don't see the point as the other half of the screen is blank. Please return the format to the full screen.</p></blockquote> <p>Christian (Galaxy Nexus):</p> <blockquote><p>If it aint broke... The new interface is AWFUL. This software was amazing until the publisher decided to give it a new interface. Three stars at best now...please give us back the old interface!!</p></blockquote> <p>Manuel (LGE Optimus One):</p> <blockquote><p>Please change to other version The previous version was better and I liked the black background much better. Please change to previous version.</p></blockquote> <p>Eddy (Galaxy S):</p> <blockquote><p>Update Sucks Don't like the version since latest update half screen scores are just plain stupid.</p></blockquote> <p>Anthony (Droid RAZR):</p> <blockquote><p>New update sucks This was a nice clean app before the update. Now it is a visual mess. Please revert back to the old style.</p></blockquote> <p>At least it wasn't <strong>all</strong> bad.</p> <p>Sam (EVO Shift 4G):</p> <blockquote><p>Love it</p></blockquote> <p>That's ... pretty much the only positive feedback I've gotten.</p> <p>This is what the old menu looked like:</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-23-08-27-59.png" title="Old slide-out menu" alt="Old slide-out menu" /></p> <p>And here's what it looks like now, in version 2.1:</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-26-11-21-22.png" title="New slide-out menu" alt="New slide-out menu" /></p> <p>Did I say 2.1? Yup. That's just been released.</p> <p>I want to find some balance between being responsive to feedback and sticking to my own sensibilities. I know that people respond angrily to change, even if the change is better -- but also that if everyone thinks something is bad, they just might be right.</p> <p>I feel that this tweak, just extending the schedule so it covers the whole screen (on a phone, anyway), will be enough to staunch some of the flow of bad reviews. I'm sure there will be more things I need to change, to keep my 53,240 users as happy with MLB Scoreboard as I am.</p> <p>If you want to see what everyone is so mad about, <a href="">go download MLB Scoreboard 2.1 now</a>!</p> Recursively Passing Blocks 2012-06-24T00:00:00-07:00 <p>Everyone knows how to use Ruby's blocks. <code>map</code> and <code>each</code>, etc, are among the first things you ever see.</p> <p>Writing a method that takes a block is something you'll probably eventually have to do, but isn't often covered. So here's a degenerate example on the way to doing what we really want:</p> <pre><code>def foo_prepender(target, &amp;block) "foo:#{target}" end foo_prepender("bar") do |thing| puts thing end =&gt; "foo:bar" </code></pre> <p>Our <code>foo_prepender</code> method takes a parameter and a block, and it calls the block with one parameter. In our example, we call it with a block that just prints out the result.</p> <p>A while back, I wrote a program that would hunt through a directory (and all its subdirectories) looking for image files to process; I was trying to finally get a handle on my photo collection, which had become more important* now that there are about 15000 pictures of my son sitting on my computer.</p> <p><em>* I should say that it's important to my girlfriend, which makes it very important to me. I've lost photo collections in the past and it hasn't bothered me so much ... if it happens this time it's not going to be great.</em></p> <p>Well, when you're dealing with directories on the filesystem, recursion is your friend. Since I wanted to be able to look at all files within a directory and all its subdirectories, I was going to need recursion. And since I wanted a way to do that without assuming anything about image processing, I wanted the image processing code to be in a block ... and the block would have to be called recursively.</p> <pre><code>def to_all_files(source_path, &amp;block) source = source.entries. select{|x| x[0]!="."}. map{|x| File.absolute_path(x, source.path)}.each do |file| if File.file?(file) file elsif to_all_files(, &amp;block) end end end </code></pre> <p>I pass in the path (or a Dir object, that'll work too), along with the block. I look at the absolute path of everything in there, and if it's a file, I pass it along to the block normally (like I showed above).</p> <p>But if it's a directory, I call this same method recursively, passing the subdirectory and a reference to the block.</p> <p>When I originally wrote this, I hoped that it'd be easily reusable. This weekend, I had to write another, different program that did some work on the files within a directory, and it turned out that this method was indeed very reusable. So I packaged it up and released it on Github as <a href="">dir-util</a>, so you can go ahead and use it too if you like.</p> MLB Scoreboard 2.0 2012-06-23T00:00:00-07:00 <p><a href="">MLB Scoreboard</a> has been by far my most popular app; as of today, there are 52,828 active users. This is far, far more popular than I'd ever imagined it could get. I still vividly remember the day it passed 1000 users, and how that seemed like an impossibly large number. Now, I remember the (much more recent) day my <em>daily</em> downloads dropped below 1000, and how bad that felt.</p> <p>Why bring it up now? Because the app has gotten a little bit crufty over the last year. When I originally released it, the app was literally just a scoreboard -- it showed you the linescores of all the games, but you couldn't click through to get any detail. Over time, I added all the features that people seem to like: individual game detail pages, play by play, video highlights, boxscores, individual player stat pages, and a live scoring widget. What had been a very fast and simple app had gotten a little bit less fast, and quite a bit less simple. And the interface started to seem a little stale to me.</p> <p>So today, I'm very happy to release <a href=""><strong>MLB Scoreboard 2.0</strong></a>, which simplifies some things and fancies up the interface.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-23-08-27-59.png" title="New slide-out menu" alt="New slide-out menu" /></p> <p>The first thing you'll probably notice is the new slide-out menu system, where the daily schedule lives. You can still page back and forth through the days, and jump to a specific day; but now, the schedule isn't its own Activity, so when you choose a game the schedule just slides away until you bring it back. This, I think, makes things seem smoother and faster.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-23-08-28-20.png" title="Game info" alt="Game info" /></p> <p>As you can see, if you've been a regular user of MLB Scoreboard, everything is still here, but is just a little bit different. The full linescore has been moved into the info tab (instead of always living above the tabs), which gives more space to all the other tabs. Instead of being white text on a black background, it's now black text on a white background, which is something a few people complained about; I think it looks better.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-23-08-28-37.png" title="Play by play" alt="Play by play" /></p> <p>The play by play gets a big benefit from the extra space, and also seems more readable because of the new color scheme.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-23-08-28-46.png" title="Batting boxscore" alt="Batting boxscore" /></p> <p>You can get the menu to slide back out either by clicking the logo in the top left corner of the screen (like how the "ActionBar" works on Android 3.0+, but this isn't actually an ActionBar because I still want it to work on Android 2.1, 2.2, and 2.3), or by clicking the back button when you're viewing an individual game.</p> <p><img src="/wp-content/uploads/mlb-scoreboard/Screenshot_2012-06-23-08-29-09.png" title="Video highlights" alt="Video highlights" /></p> <p>The video highlights are still here.</p> <p>Hopefully everybody likes the new interface, and I get a bunch of new downloads before MLB takes heed and I <a href="">go the way of Batter's Box</a>.</p> <p>So <a href="">get it while it's hot</a>!</p> Cocos2d - Drag from the point you're touching 2012-05-31T00:00:00-07:00 <p>Lately, I've been making apps for kids -- having a kid makes you do that, apparently -- and a big part of making it fun for them is to let them drag stuff around the screen. The way I'd been doing it was in my <code>ccTouchesBegan</code> method I'd detect which object has been touched and mark it as the "current" object, and in my <code>ccTouchesMoved</code> method I'd set the object's position to the current touch position.</p> <p>For naively dragging things around, that works. But the "position" of my objects is their center, which means that as soon as you start dragging the object, it jumps such that your finger is right in the middle of it. Having your finger in the middle of the object isn't a problem at all, especially while dragging, but that initial jump -- which can be kind of a long way if you have a big object to drag -- is pretty crappy.</p> <p>So, what I want to do is calculate the distance from where you're touching to the center of the object, and then set the position of the object such that it takes that difference into account.</p> <p>First, I'll edit <code>ThingToBeDragged.h</code>:</p> <pre><code>@property(nonatomic, assign) CGPoint dragDifference; -(void) setDragPosition:(CGPoint)dragPosition; -(void) dragTo:(CGPoint)location; </code></pre> <p>The <code>dragDifference</code> variable is where I'll store the difference between where the user touched the object and the object's center. <code>setDragPosition</code> is where that calculation is made (you should call this when the touch begins), and <code>dragTo</code> actually does the move (called when the touch moves).</p> <p>In <code>ThingToBeDragged.m</code> (remember to <code>@synthesize dragDifference</code> too):</p> <pre><code>-(void) setDragPosition:(CGPoint)dragPosition { self.dragDifference = ccpSub([self position], dragPosition); } -(void) dragTo:(CGPoint)location { self.position = ccpAdd(location, [self dragDifference]); } </code></pre> <p>That's the meat of the thing. When the touch begins, we subtract the point where they touched (<code>dragPosition</code>) from the center of our object. Then, when it's time to move the object, we add that difference back onto the <code>location</code> they're currently touching and set that as the new center of the object.</p> <p>Over in <code>GameLayer.m</code>:</p> <pre><code>-(void) ccTouchesBegan:(NSSet *)touches withEvent:(UIEvent *)event { CGPoint location = [self convertTouchToNodeSpace:[touches anyObject]]; // ... a bunch of stuff determining which thing was touched ... [self currentThing].dragPosition = location; } -(void) ccTouchesMoved:(NSSet *)touches withEvent:(UIEvent *)event { CGPoint location = [self convertTouchToNodeSpace:[touches anyObject]]; if ([self currentThing] != nil) { [[self currentThing] dragTo:location]; } } </code></pre> <p>And now, when you start dragging something around, if you put your finger somewhere other than the center of the object, your finger will stay on that part of the object the entire time you're dragging it. (The thing doesn't jump such that your finger is always in the center of the object.)</p> <p>Makes things better, and is pretty easy.</p> Introducing rest/api 2012-05-18T00:00:00-07:00 <p>I have recently been building a modular service-oriented architecture, using Sinatra to create simple, focused HTTP services. Part of the benefit of that is that you can consume one service within another, or consume multiple services in a wholly separate application (and if you find that you need to, you can scale the services separately).</p> <p>But in order for that to work nicely, you need to be able to call those services easily. So I wrote a little class that wraps the HTTP GET/POST calls, translates to/from JSON as needed, and gives you nice Ruby objects to work with.</p> <p>For example, say you have a service with the following endpoints:</p> <ul> <li>/thing/:thing_id</li> <li>/thing/create</li> <li>/thing/:thing_id/update</li> <li>/other_thing/:other_thing_id</li> <li>/other_thing/create</li> <li>/other_thing/:other_thing_id/update</li> </ul> <p>And you want to consume that service in your program. You <strong>could</strong> spin up HTTP requests every time you want to call them, but that'd kind of suck.</p> <p>Or, you could use <a href=""><strong>rest-api</strong></a>.</p> <pre><code>module MyThings::API class Core include REST::API::Base attr_reader :thing, :other_thing def initialize @thing = @other_thing = end end end class MyThings::API::Core::Thing def initialize(api) @api = api end def get(thing_id) @api.GET("/thing/#{thing_id}") end def create(param1, param2) @api.POST("/thing/create", { "param1" =&gt; param1, "param2" =&gt; param2 }) end def update(thing_id, param1, param2) @api.POST("/thing/#{thing_id}/update", { "param1" =&gt; param1, "param2" =&gt; param2 }) end end class MyThings::API::Core::OtherThing def initialize(api) @api = api end def get(other_thing_id) @api.GET("/other_thing/#{other_thing_id}") end def create(param1, param2) @api.POST("/other_thing/create", { "param1" =&gt; param1, "param2" =&gt; param2 }) end def update(other_thing_id, param1, param2) @api.POST("/other_thing/#{other_thing_id}/update", { "param1" =&gt; param1, "param2" =&gt; param2 }) end end </code></pre> <p>To use it, you just need to instantiate your Core API class and set its base_url field (basically, that's where your service is running).</p> <pre><code>core = core.base_url = "http://localhost:4567" thing = core.thing.get(thing_id) puts thing.param1 other_thing = core.other_thing.create(param1, param2) puts other_thing.param1 </code></pre> <p>And let's say you had another service that you wanted to be able to use in your app (ie, hosted on a different server). You'd create another class that includes <code>REST::API::Base</code> and hook up your routes, and could just do:</p> <pre><code>core2 = core2.base_url = "http://localhost:4568" another_thing = core2.another_thing.get(another_thing_id) </code></pre> <p>I'm sure I'll change some things around as I find ways to make it even easier (or, more likely, as I find things about it that suck). But for now, it's good enough for me to use in my projects.</p> <p>Meanwhile, if you're interested, you can <a href="">grab it from Github</a>.</p> A Rake task to create Mongoid indexes 2012-05-16T00:00:00-07:00 <p>I've recently started using Mongoid behind my Sinatra-based web services, and I like it. But I wanted an easy way to create indexes on my Mongo collections, and the options seemed to be a) use Rails, or b) set "autocreate_indexes" to true in mongoid.yml. I don't want to do either of those things.*</p> <p><em>* Notably, "autocreate_indexes" can be unwise because creating indexes can take a while and you don't necessarily want that delay while your service is starting up. It's better to take that hit out of band, like in a Rake task.</em></p> <p>Now, Mongoid provides a Rake task to create your indexes if you happen to be using Rails. But if you're not, what can you do? Well, write your own:</p> <pre><code>namespace :db do task :create_indexes, :environment do |t, args| unless args[:environment] puts "Must provide an environment" exit end yaml = YAML.load_file("mongoid.yml") env_info = yaml[args[:environment]] unless env_info puts "Unknown environment" exit end Mongoid.configure do |config| config.from_hash(env_info) end MyModels::Thing1.create_indexes MyModels::Thing2.create_indexes MyModels::Thing3.create_indexes end end </code></pre> <p>And you use it as you'd expect:</p> <pre><code>rake db:create_indexes[development] </code></pre> <p>You have to specify the environment, because Mongoid will choose which database to connect to based on that. It supports "development", "test", and "production", which you have to specify in your mongoid.yml file.</p> <p>It's working well for me so far on two projects. One thing I wish is that I didn't have to list all the model classes, that I could have that determined automatically. Maybe I'll figure that out later.</p> JMock: Mocking Concrete Classes 2012-05-15T00:00:00-07:00 <p>Normally when you're testing with JMock, you want to design your code to use interfaces as much as possible, and mock the interfaces; that way, the only implementation code that's involved in your tests is the code you actually want to test.</p> <p>But sometimes, you find yourself using something that doesn't have an interface, but you don't want to include its functionality within your test. You just want to be able to mock it. JMock normally doesn't let you do that. If you try, you get something like this (a test failure):</p> <pre><code>java.lang.IllegalArgumentException: com.vikinghammer.sample.SomeConcreteClass is not an interface at java.lang.reflect.Proxy.getProxyClass( at java.lang.reflect.Proxy.newProxyInstance( at org.jmock.lib.JavaReflectionImposteriser.imposterise( at org.jmock.Mockery.mock( at org.jmock.Mockery.mock( </code></pre> <p>I've done this before, a couple of times, but now that I find I need to do it again I'd forgotten exactly how to do it. Maybe writing it down will help me remember, for next time! (<a href="">JMock's website explains how to do it</a>, but I thought I'd show how I do it in my environment, which is slightly different.)</p> <p>First, add the necessary dependency to your pom.xml (that is, if you're using Maven).</p> <pre><code>&lt;dependency&gt; &lt;groupId&gt;org.jmock&lt;/groupId&gt; &lt;artifactId&gt;jmock-legacy&lt;/artifactId&gt; &lt;version&gt;2.5.1&lt;/version&gt; &lt;scope&gt;test&lt;/scope&gt; &lt;/dependency&gt; </code></pre> <p>Then, you'll need to import your <code>ClassImposteriser</code>.</p> <pre><code>import org.jmock.lib.legacy.ClassImposteriser; </code></pre> <p>And then:</p> <pre><code>Mockery context = new JUnit4Mockery(); @Before public void setupMocks() { context.setImposteriser(ClassImposteriser.INSTANCE); concreteThing = context.mock(SomeConcreteClass.class); } </code></pre> <p>And now you can use it just like you would any mocked interface. Nice and simple.</p> Android: Use AlarmManager instead of a Service with a TimerTask 2012-04-22T00:00:00-07:00 <p>In building the live scoring widget for MLB Scoreboard, I needed to be able to reload the scores, repeatedly, when the app isn't open. Since Android lets you start a Service that runs in the background, that seemed like the best way to go.</p> <p>I started the Service both when your phone first booted and each time you opened the app (to make sure it was always available in the background). I used a TimerTask and a Handler to run the "download the scores and update the widget" code periodically; I limited how often it actually did the downloading work by checking when the game actually started and only downloading scores once per day (in the morning) when the game wasn't playing, but then loading it every minute during the game. I thought that would help save the battery a little bit.</p> <p>I was wrong. Because of the way TimerTask works in a Service, it was still executing code very frequently even thought it wasn't actually downloading anything. As a result, the phone kept waking up to check if it should download anything, which meant it could never sleep long enough to save any battery. With the early prototype of my widget sitting on my phone, I ran out of power even more quickly than my battery's waning capacity normally would.</p> <p>Enter <a href=""><strong>AlarmManager</strong></a>. Rather than starting a Service that runs in the background always, <em>this</em> is what you want to use to schedule your code to run when your app isn't open.</p> <p>Crucially, it gives you the option of waking up the phone to execute your code or just <em>waiting until the user wakes up the phone</em> before executing the scheduled code.</p> <p>Here's how you schedule a receiver.</p> <pre><code>Intent alarmIntent = new Intent(context, FavoriteTeamDownloadReceiver.class); PendingIntent pendingIntent = PendingIntent.getBroadcast(context, 0, alarmIntent, PendingIntent.FLAG_UPDATE_CURRENT); AlarmManager alarmManager = (AlarmManager)context.getSystemService(Context.ALARM_SERVICE); alarmManager.set(AlarmManager.RTC, Calendar.getInstance().getTimeInMillis(), pendingIntent); </code></pre> <p><strong>Note:</strong> The code I want to run is a <strong>BroadcastReceiver</strong> called <strong>FavoriteTeamDownloadReceiver</strong>. By using PendingIntent.FLAG_UPDATE_CURRENT, if I schedule another Intent before the previous one executes, the extras in its bundle will be overridden with the new values. That's not important here, but it's what I want usually.</p> <p>You grab the AlarmManager by calling <strong>getSystemService(Context.ALARM_SERVICE)</strong>, you never want to instantiate it yourself.</p> <p>And by scheduling it with <strong>AlarmManager.RTC</strong>, I can specify the time it should run but if the phone is sleeping, the intent will not run <em>until the phone is woken up</em> by some other means. It doesn't wake up to run my code. (If you <em>want</em> it to wake up to run your code, like if you were actually making an alarm clock or something, you could use <strong>AlarmManager.RTC_WAKEUP</strong>.)</p> <p>Of course, right now, this wouldn't work. You need to actually write your BroadcastReceiver:</p> <pre><code>public class FavoriteTeamDownloadReceiver extends BroadcastReceiver { @Override public void onReceive(final Context context, Intent intent) { // do the thing in here // including figuring out the next time you want to run // and scheduling another PendingIntent with the AlarmManager } } </code></pre> <p>And in your <strong>AndroidManifest.xml</strong> file, you need to tell the system that you're going to be doing this:</p> <pre><code>&lt;receiver android:name=".service.FavoriteTeamDownloadReceiver" android:process=":remote" /&gt; </code></pre> <p>Now you're up and running. When your phone is in your pocket, nothing will happen; it doesn't wake up to check if it should download anything, and doesn't use any extra battery. When the phone wakes up, it will run the most recently scheduled intent that should have executed, and by the time you unlock the phone your home screen widget will already have been updated.</p> <p>Pretty useful. And this seems like the right way to do it.</p> Connecting Highcharts and jqGrid with trigger and bind 2012-02-02T00:00:00-08:00 <p>I have a page where I'm using <a href="">Highcharts</a> for a graph, and jqGrid for a grid; they're showing different views of similar data, but are sourced from two different calls to the backend. The graph is a summary, and the grid is detailed data, and it doesn't make much sense for that to come back in one call.</p> <p>In my graph, I have several lines; Highcharts lets the user show/hide those lines manually. I want to make the data in the grid reflect what's been shown or hidden in the graph.</p> <p>I first tried to bind the Highcharts series via Knockout, somehow, to what should be shown in the grid. Maybe it's possible, but I couldn't figure out how to get Knockout to bind to variables deep inside the Highcharts object structure.</p> <p>But Javascript and jQuery have event listeners, right? Hopefully Highcharts is nice enough to use those, so I can get an event when the series visibility is changed.</p> <p>I looked in the Highcharts source, and down in the Series' <strong>setVisibility</strong> method (more than 10000 lines deep), I found this:</p> <pre><code>fireEvent(series, showOrHide); </code></pre> <p>(The "showOrHide" variable is always either "show" or "hide".)</p> <p>That looks promising, but what does fireEvent do?</p> <pre><code>fireEvent = function (el, type, eventArguments, defaultFunction) { var event = jQ.Event(type), detachedType = 'detached' + type; extend(event, eventArguments); // Prevent jQuery from triggering the object method that is named the // same as the event. For example, if the event is 'select', jQuery // attempts calling and it goes into a loop. if (el[type]) { el[detachedType] = el[type]; el[type] = null; } // trigger it jQ(el).trigger(event); // attach the method if (el[detachedType]) { el[type] = el[detachedType]; el[detachedType] = null; } if (defaultFunction &amp;&amp; !event.isDefaultPrevented()) { defaultFunction(event); } }; </code></pre> <p>Most of that is necessary stuff that I don't care about, but the key line is this one:</p> <pre><code>jQ(el).trigger(event); </code></pre> <p>That means that if I grab the Series object and wrap it with a jQuery call, I can call "bind" on it and be notified any time "trigger" is called. So I did that, for each series object:</p> <pre><code>var listener = function() { MyGrid.updateGrid(MyGraph.visibleSeries()); }; for (var i=0; i &lt; this.chart.series.length; i++) { $(this.chart.series[i]).bind("show", listener); $(this.chart.series[i]).bind("hide", listener); } </code></pre> <p>I created a function and saved it to a variable, and bound it to the "show" and "hide" events on each of my Series. I could have passed it in as an anonymous function, but I would have had to define it twice. I could have created it as an actual function on my object, but this way it's scoped right here and won't be accessible anywhere else.</p> <p>On my MyGraph object, I need to be able to check which series are currently visible:</p> <pre><code>function visibleSeries() { var selected = []; for (var i=0; i &lt; MyGraph.chart.series.length; i++) { if (MyGraph.chart.series[i].visible) { selected.push(MyGraph.chart.series[i].name); } } return selected; } </code></pre> <p>That returns a list of series names, which I can then pass to the MyGrid object to update which lines it should display:</p> <pre><code>var updateGrid = function(events) { var newData = {}; = 1; = 1; newData.rows = []; for (var i=0; i &lt; MyGrid.gridData.rows.length; i++) { if ($.inArray(MyGrid.gridData.rows[i].cell[1], events) != -1) { newData.rows.push(MyGrid.gridData.rows[i]); } } newData.records = newData.rows.length; MyGrid.updateData(JSON.stringify(newData)); }; </code></pre> <p>I've stored "gridData" in MyGrid, which is all the data for the grid that was returned from the server. It doesn't get updated here, which means I can look at the full dataset again each time the visible series changes, and update the grid accordingly.</p> <p>Victory! Now, whenever the user hides a series in the graph, it disappears from the grid. When they add it back, it reappears in the grid. The grid also updates its display of total records available, and the number of pages, and always jumps back to the first page.</p> Cassandra Counter Columns 2012-01-26T00:00:00-08:00 <p>Cassandra supplies "counter columns", which are used to store a number that incrementally counts a value. You might use a counter to keep track of pageviews or other events.</p> <p>I couldn't find much documentation about how to use these, especially through the CLI (which is where I typically start out when trying to investigate a new feature). So here goes with an explanation of what I've found.</p> <p>Create your "counter" column family:</p> <pre><code>create column family MyCounters with default_validation_class=CounterColumnType and comparator=UTF8Type; </code></pre> <p>Note that you don't <em>have</em> to create the column family with a "default_validation_class" of CounterColumnType, but by doing so you get a few major advantages:</p> <ol> <li>You can create arbitrary column names as counters, rather than having to add them explicitly and being required to know ahead of time what you'll be counting</li> <li><p>I couldn't figure out how to actually do that, despite the fact that the documentation I read seemed to indicate it should be possible; instead, when I attempted to <strong>update column family</strong> with some column_metadata, it failed, like so:</p> <pre><code> [default@MyTest] update column family MyCounters with column_metadata = [{column_name:thing-1, validation_class:CounterColumnType}]; org.apache.thrift.TApplicationException: Internal error processing system_update_column_family </code></pre></li> </ol> <p>That's all the information you get in the CLI. Apparently, when you see "Internal error", that means you need to look in <strong>/var/log/cassandra/system.log</strong>, where you'll see:</p> <pre><code>ERROR [pool-2-thread-1] 2012-01-26 09:41:01,705 (line 4038) Internal error processing sys tem_update_column_family java.lang.RuntimeException: org.apache.cassandra.config.ConfigurationException: Cannot add a counter col umn (thing-1) in a non counter column family at org.apache.cassandra.thrift.CassandraServer.system_update_column_family( 1049) at org.apache.cassandra.thrift.Cassandra$Processor$system_update_column_family.process(Cassandra .java:4032) at org.apache.cassandra.thrift.Cassandra$Processor.process( at org.apache.cassandra.thrift.CustomTThreadPoolServer$ .java:187) at java.util.concurrent.ThreadPoolExecutor$Worker.runTask( at java.util.concurrent.ThreadPoolExecutor$ at Caused by: org.apache.cassandra.config.ConfigurationException: Cannot add a counter column (thing-1) in a non counter column family at org.apache.cassandra.config.CFMetaData.validate( at org.apache.cassandra.config.CFMetaData.fromThrift( at org.apache.cassandra.thrift.CassandraServer.system_update_column_family( 1045) ... 6 more </code></pre> <p>So that's why it's important to <strong>create column family</strong> as a "counter column family" from the start. Because of this, I intend to use these counter column families as what are essentially extra, metadata-only denormalized column families; ie, I won't be mixing actual data in with these counters. Instead, the actual data will live elsewhere, and the counters will live apart, by themselves. Remember, in Cassandra, this sort of denormalization is considered not only acceptable, but required.</p> <p>Okay, enough of that diversion. Let's continue with using the CLI to make and use our counter column family.</p> <p>I usually tell the CLI to use UTF-8, rather than raw hex bytes (I don't know why it defaults to expecting you, a human, to be able to read and write raw bytes):</p> <pre><code>assume MyCounters keys as utf8; </code></pre> <p>Now you can use <strong>incr</strong> to increment your counters:</p> <pre><code>incr MyCounters['key-1']['thing-1']; incr MyCounters['key-1']['thing-2']; incr MyCounters['key-1']['thing-1']; incr MyCounters['key-2']['thing-1']; incr MyCounters['key-2']['thing-3']; incr MyCounters['key-2']['thing-3']; incr MyCounters['key-1']['thing-4'] by 5; </code></pre> <p>And you can pull out the whole thing with <strong>list</strong>:</p> <pre><code>[default@MyTest] list MyCounters; Using default limit of 100 ------------------- RowKey: key-1 =&gt; (counter=thing-1, value=2) =&gt; (counter=thing-2, value=1) =&gt; (counter=thing-4, value=5) ------------------- RowKey: key-2 =&gt; (counter=thing-1, value=1) =&gt; (counter=thing-3, value=2) 2 Rows Returned. Elapsed time: 8 msec(s). </code></pre> <p>Or you can <strong>get</strong> the values for a single RowKey:</p> <pre><code>[default@MyTest] get MyCounters['key-1']; =&gt; (counter=thing-1, value=2) =&gt; (counter=thing-2, value=1) =&gt; (counter=thing-4, value=5) Returned 3 results. Elapsed time: 8 msec(s). </code></pre> <p>You can also <strong>get</strong> the value of a single counter within a column:</p> <pre><code>[default@MyTest] get MyCounters['key-1']['thing-1']; =&gt; (counter=thing-1, value=2) Elapsed time: 8 msec(s). </code></pre> <p>As you can see, counter columns are not actually complicated or difficult to use. This got me around some of the initial issues I encountered, so hopefully it helps someone else.</p> <p><strong>One last thing</strong></p> <p>When using counters, there are <a href="">some things to consider</a> when it comes to your consistency level:</p> <blockquote><p>it’s important to understand that unlike normal columns, a write to a counter requires a read in the background to ensure that distributed counter values remain consistent across replicas. If you write at a consistency level of ONE, the implicit read will not impact write latency, hence, ONE is the most common consistency level to use with counters.</p></blockquote> <p>I haven't addressed this yet, but I'll keep it in mind.</p> Using Dependo to help with testing 2012-01-08T00:00:00-08:00 <p>Last week, I <a href="">introduced Dependo</a>, my new dependency injection framework for Ruby.</p> <p>Today, I want to demonstrate something that it makes very easy -- and something that I wouldn't know how to adequately test without it.</p> <p>In our OCSP responder, we rely on a Redis database and can configure (at startup) whether we want to copy the "nonce"* from the request into the response.</p> <p><em>* The idea of the "nonce" is that it increases the browser's ability to trust a response from the server, because not only is the response signed, the signed response contains the timestamp that was given in the request. It makes it much, much more difficult for a replay attack to work against the service. This is mostly irrelevant to my discussion of Dependo, I just wanted to explain it (a little) so you're not too confused.</em></p> <p>But in our unit tests, we need to mock the Redis database (because we don't want to have to actually run a database in order for our tests to pass) and re-configure that "copy nonce" behavior between tests.</p> <p>First, before each test, we set up our <strong>Dependo::Registry</strong> with the values we're going to want. We clear it out from the previous test, set the Logger to not log anything (we can change that if we want to see logging to help figure something out, but generally we don't want to log anything during tests), mock Redis, set a default "copy nonce" setting (defaults to false here), and read our config file (I'd really rather not do it like this, we'll get around to it later, I hope).</p> <pre><code>before :each do # clear the dependo before each test Dependo::Registry.clear Dependo::Registry[:log] = # we always want to mock with a new redis @redis = double("redis") Dependo::Registry[:redis] = @redis # default value for :copy_nonce is false (can override on a per-test basis) Dependo::Registry[:copy_nonce] = false # read the config.yaml Dependo::Registry[:config_pool] = R509::Config::CaConfigPool.from_yaml("certificate_authorities","config.yaml")) end </code></pre> <p>Now, we haven't finished setting up each test. In our <strong></strong>, we also add an R509::Ocsp::Signer object to the Dependo::Registry. The Ocsp::Signer is constructed from the other items in the Dependo::Registry that are configured, in production, in the file. So where did we put it?</p> <pre><code>def app # this is executed after the code in each test, so if we change something in the dependo registry, it'll show up here (we will set :copy_nonce in some tests) Dependo::Registry[:ocsp_signer] = :configs =&gt; Dependo::Registry[:config_pool].all, :validity_checker =&gt;[:redis]), :copy_nonce =&gt; Dependo::Registry[:copy_nonce] ) R509::Ocsp::Responder end </code></pre> <p>I define it in my spec file's #app method because of the order of execution. That order is:</p> <ol> <li>The code in "before :each"</li> <li>The code in the individual test</li> <li>The code in #app</li> </ol> <p>Why is that relevant?</p> <p>Because, in the case where we want to override the default value of Dependo::Registry[:copy_nonce], we need to be able to do that in the individual test code, and then read it in the #app method where the R509::Ocsp::Signer is built.</p> <pre><code>it "copies nonce when copy_nonce is true" do @redis.should_receive(:hgetall).with("cert:/C=US/ST=Illinois/L=Chicago/O=Ruby CA Project/CN=Test CA:872625873161273451176241581705670534707360122361").and_return({"status" =&gt; R509::Validity::VALID}) # set to true for this test (this works because the app doesn't get set up until after this code) Dependo::Registry[:copy_nonce] = true get '/MHsweTBSMFAwTjAJBgUrDgMCGgUABBQ4ykaMB0SN9IGWx21tTHBRnmCnvQQUeXW7hDrLLN56Cb4xG0O8HCpNU1gCFQCY2eXAtMNzVS33fF0PHrUSjklF%2BaIjMCEwHwYJKwYBBQUHMAECBBIEEDTJniOQonxCRmmHAHCVstw%3D' request ="MHsweTBSMFAwTjAJBgUrDgMCGgUABBQ4ykaMB0SN9IGWx21tTHBRnmCnvQQUeXW7hDrLLN56Cb4xG0O8HCpNU1gCFQCY2eXAtMNzVS33fF0PHrUSjklF+aIjMCEwHwYJKwYBBQUHMAECBBIEEDTJniOQonxCRmmHAHCVstw=")) ocsp_response = R509::Ocsp::Response.parse(last_response.body) request.check_nonce(ocsp_response.basic).should == R509::Ocsp::Request::Nonce::PRESENT_AND_EQUAL end </code></pre> <p>Without Dependo (or a framework with similar capability), it'd be more difficult to test this behavior. Frankly, I don't know if you could even do it, much less do it elegantly.</p> <p>So the first time a tough spot like this came up, I found myself glad to have <a href="">Dependo</a>.</p> Ruby Dependency Injection: Introducing Dependo 2012-01-04T00:00:00-08:00 <p>Today, I discovered that I need Dependency Injection in Ruby. Other people who are probably smarter than I am <a href="">have said you don't need it</a>, because Ruby is so dynamic and you can just do whatever you want to without having the structure provided by dependency injection.</p> <blockquote><p><strong>DI frameworks are unnecessary.</strong> In more rigid environments, they have value. In agile environments like Ruby, not so much. The patterns themselves may still be applicable, but beware of falling into the trap of thinking you need a special tool for everything. Ruby is Play-Doh, remember! Let’s keep it that way.</p></blockquote> <p>Others have apparently taken that to mean that dependency injection is simply unnecessary in Ruby, despite the fact that that's explicitly not what Jamis Buck is saying here.</p> <blockquote><p>So, is there no room for DI in Ruby? There definitely is. I use DI nearly every day in Ruby, but I do not use a DI framework. Ruby itself has sufficient power to represent any day-to-day DI idioms you need.</p></blockquote> <p>He talks about the ways he injects his dependencies, using Ruby without any framework:</p> <ul> <li>Factory method that takes an optional class and instantiates an object for you</li> <li>A second factory method that you can override in a subclass for testing</li> <li>Pass in either classes or implementations in a constructor</li> </ul> <p>But let's say you've got the following problem:</p> <ul> <li>You're writing a web app, using Sinatra (or anything else, really)</li> <li>You're using an ORM library, like Sequel <ul> <li>Your models have methods on them</li> </ul> </li> <li>You want some sort of logging <ul> <li>You want to define just one Logger object and use it in your Sinatra app and your Sequel models</li> </ul> </li> </ul> <p>Passing the Logger instance from the Sinatra app to the Sequel models either in a constructor or in each method is an ugly hack, and a non-starter. So what do you do?</p> <p>Enter: <a href="">Dependo</a>!</p> <p>Dependo lets you register your Logger object. I do this in my</p> <pre><code>require "dependo" Dependo::Registry[:log] = </code></pre> <p>Then, you can just include everything in the Registry as methods in your Sinatra app, and use them as if they're instance methods:</p> <pre><code>class MyApp &lt; Sinatra::Base include Dependo::Mixin get "/?" do "I'm logging!" "Hello, world." end end </code></pre> <p>You can also include the methods in your Sequel models:</p> <pre><code>class MyThing &lt; Sequel::Model include Dependo::Mixin extend Dependo::Mixin def self.do_some_query "I'm querying with a class method" self end def perform_some_other_action "I'm calling an instance method" end end </code></pre> <p>Note that in the model class, we used both <strong>include</strong> and <strong>extend</strong>. We do that so we can get the Dependo functionality in both instance methods (via include) <em>and</em> class methods (via extend).</p> <p>Now, the Dependo::Registry doesn't care what you put in it. You've already seen that we can put an object in it (like a Logger, as above). You can also put in a number,</p> <pre><code>Dependo::Registry[:my_important_number] = 7 </code></pre> <p>or a string,</p> <pre><code>Dependo::Registry[:my_important_string] = "seven" </code></pre> <p>or even a function,</p> <pre><code>Dependo::Registry[:my_important_proc] = { |x| x + 7 } </code></pre> <p>or a lambda,</p> <pre><code>Dependo::Registry[:my_important_lambda] = lambda { |x| x * 7 } </code></pre> <p>and you'll use those like so:</p> <pre><code>class DemoClass include Dependo::Mixin def do_things puts my_important_number puts my_important_string puts puts end end </code></pre> <p><em>* I don't know how to call a Proc or a lambda in this case without using #call on them. I'd much rather the syntax be something like "my_important_lambda(6)" ... so, does anybody have any ideas? Thanks!</em></p> <p>Do I think dependency injection is necessary, even in Ruby? I do. It helps with separation of concerns, so each of your classes can do only the things it cares about; it helps with unit testing, so you can mock your classes' dependencies rather than test them all the way through (as you might in an integration test). You also don't have to muck up your constructors or method definitions; your code's API can stay the way you want it, and having to "include" and/or "extend" a mixin is a small price to pay (I think) for what you get.</p> <p>I'm using Dependo with some success. Let me know if you do too -- or, more importantly, if you find something about it that sucks, so I can try to fix it.</p> Ruby OpenSSL::X509::Name throws away unknown subject component names?! 2011-12-21T00:00:00-08:00 <p>OpenSSL::X509::Name is a class in Ruby's OpenSSL bindings that lets you deal with the subject line of SSL certificates. It's useful and necessary, though dealing with it can be kind of annoying. But as of Ruby 1.9.3, there's a big bug that threatens to be a deal-breaker for anyone doing significant SSL work in Ruby: its handling of undefined OIDs.</p> <p>You're accustomed to dealing with subject components by their shortname, things like "CN" or "O" or "C", etc. But underneath the shortname is a different representation; the longname can be something like "" instead.</p> <p>Now, you're allowed to create an OpenSSL::X509::Name object with an unknown OID like this.</p> <pre><code>name = [["CN", ""], ["", "US"]] </code></pre> <p>And if you now get the subject line, all is well.</p> <pre><code>name.to_s #=&gt; "/" </code></pre> <p>But sometimes you want to get that array (like the one you passed into the constructor) back out and deal with it directly. We're doing that in one of our projects, where we want to wrap OpenSSL::X509::Name in a friendlier interface. But what happens if you do it?</p> <pre><code>name.to_a #=&gt; [["CN", "", 12], ["UNDEF", "US", 12]] </code></pre> <p><strong>UNDEF</strong>?! What help is that, you guys? I mean, after all ...</p> <pre><code> name.to_a #OpenSSL::X509::NameError: invalid field name # from (irb):8:in `initialize' # from (irb):8:in `each' # from (irb):8:in `initialize' # from (irb):8:in `new' # from (irb):8 </code></pre> <p>Yeah, you can't pass the output from OpenSSL::X509::Name#to_a back into the constructor; if you try, it'll blow up.</p> <p>But I kind of need to do that; when we parse a CSR or a certificate, it contains an OpenSSL::X509::Name, and we need to be able to read it. When we do that, we read the array out of #to_a, because parsing the subject line string out of #to_s is dangerously fragile (because the delimiters, "/" and "=", can be used unescaped in the subject component values).</p> <p>Enough complaining. What's the solution?</p> <p>We staged a two-pronged attack on this problem. The first step was to <a href="">patch Ruby</a>, to fix <a href="">the bug</a> in its OpenSSL bindings. My teammate <a href="">Paul Kehrer</a> did this.</p> <pre><code>short_name = OBJ_nid2sn(OBJ_ln2nid(long_name)); if (strcmp(short_name,"UNDEF") == 0) { return_name = &amp;long_name; } else { return_name = short_name; } </code></pre> <p>It's a fairly simple fix (once you've, you know, gotten deep enough into Ruby's C source). It used to attempt to translate the longname into a shortname, and if no translation was found, it'd just leave the shortname as UNDEF. Instead, he checks for UNDEF and if that's the shortname, he resets it to be the (original) longname. That way, the OID is preserved in the OpenSSL::X509::Name object. Victory!</p> <p>But ... that'd mean our project would only work on a custom patch of Ruby 1.9.3, and we were hoping to maintain compatibility across both 1.8 and 1.9, and we certainly can't rely on people to use Paul's patch rather than the actual release of Ruby. And even if (hopefully "when") Ruby accepts the patch, that'd mean we'd have to require Ruby (at least) 1.9.4, which is pretty close to unacceptable. We need an interim solution -- where "interim" is taken to mean "useful for at least a few years, potentially indefinitely".</p> <p>That's where the second prong of our attack comes in. Sanitizing the OpenSSL::X509::Name#to_a right in Ruby!</p> <pre><code># Sanitize an X509::Name. The #to_a method replaces unknown OIDs with "UNDEF", but the #to_s # method doesn't. What we want to do is build the array that would have been produced by #to_a # if it didn't throw away the OID. class NameSanitizer # @option name [OpenSSL::X509::Name] # @return an array of the form [["OID", "VALUE], ["OID", "VALUE"]] with "UNDEF" replaced by the actual OID def sanitize(name) line = name.to_s array = name.to_a.dup used_oids = [] undefined_components(array).each do |component| begin # get the OID from the subject line that has this value oids = line.scan(/\/([\d\.]+)=#{component[:value]}/).flatten if oids.size == 1 oid = oids.first else oid ={ |match| not used_oids.include?(match) }.first end # replace the "UNDEF" OID name in the array at the index the UNDEF was found array[component[:index]][0] = oid # remove the first occurrence of this in the subject line (so we can handle the same oid/value pair multiple times) line = line.sub("/#{oid}=#{component[:value]}", "") # we record which OIDs we've used in case two different unknown OIDs have the same value used_oids &lt;&lt; oid rescue # I don't expect this to happen, but if it does we'll just not replace UNDEF and continue end end array end private # get the components from #to_a that are UNDEF # @option array [Array&lt;OpenSSL::X509::Name&gt;] # @return [{ :index =&gt; the index in the original array where we found an UNDEF, :value =&gt; the subject component value }] def undefined_components(array) components = [] array.each_index do |index| components &lt;&lt; { :index =&gt; index, :value =&gt; array[index][1] } if array[index][0] == "UNDEF" end components end end </code></pre> <p>This solution, as you may have noticed, is a little bit more complicated. I'll try to explain.</p> <p>First, we take advantage of the fact that OpenSSL::X509::Name#to_s contains <strong>all</strong> the data we need, despite the fact that #to_a throws it away. So we loop over #to_a to find all the subject components whose name is UNDEF and record their index in the #to_a array, as well as their value (we don't need to record their names, because we know they're always UNDEF).</p> <p>Then, we loop over all those UNDEF subject components, and we determine what the original OID was from the subject line string from #to_s. For that, we take advantage of the fact that these unknown OIDs will be of the form, or something like that; ie, our regex looks for anything with digits and periods in the place of the subject component name, <em>with the same value</em> as the undefined subject component.</p> <p>Note that we also rely on the fact that #to_s and #to_a maintain the order of subject components, so as we loop over the unknown OIDs, the current one will always correspond to the first match from the subject line.</p> <p>After updating the subject component name in the array, we remove the subject component from the subject line (but only the first occurrence of it), and we record that we've used that OID. Both of those steps are to ensure that if the same OID occurs multiple times, or if two different OIDs have the same value, that we handle that properly and correctly maintain the order of the components.</p> <p>And now ...</p> <pre><code>sanitizer = sanitizer.sanitize(name) #=&gt; [["CN", "", 12], ["", "US", 12]] #OpenSSL::X509::NameError: invalid field name #=&gt; / </code></pre> <p>It works! As long as we pass the original name to the sanitizer, we can get the original OIDs in the array structure we need (the same one returned from #to_a).</p> <p>This will work across Ruby 1.8 and 1.9, and if Ruby accepts Paul's patch, it'll also work in 1.9.4; if the bug is fixed, NameSanitizer#undefined_components will not find any UNDEF subject components, and none of our complex code will ever be touched.</p> <p>I think this exercise demonstrates a few things:</p> <ul> <li>It's considerably simpler to fix this kind of thing at a lower level, if possible.</li> <li>Even if a bug in your language gets fixed, you should still try to figure out a backwards-compatible fix.</li> <li>Use open source languages! It's awesome to have the option of fixing Ruby and submitting a patch.</li> <li>If you're working on an open source project, you can write posts like this about it.</li> </ul> <p>This turned a horrifyingly annoying Tuesday evening (when Paul discovered the bug) into a pretty fun Wednesday morning (when we fixed it).</p> Mocking unit tests in Ruby, with Rspec 2 and test doubles 2011-12-13T00:00:00-08:00 <p>Mocking in your unit tests is a useful and powerful thing. My experience with it had been with JMock, in Java. But for one of my current projects I've found that I need it with rspec, in Ruby. I didn't find a ton of helpful documentation on the subject, so hopefully this explanation helps someone out.</p> <p>The concept of this class is that it looks up an SSL certificate serial number in a Redis database, and returns some status information. Those details are mostly irrelevant to the question of mocking, but I thought I'd get it out of the way.</p> <pre><code>module R509::Validity::Redis class Checker &lt; R509::Validity::Checker def initialize(redis) raise"Redis must be provided") if redis.nil? @redis = redis end def check(serial) raise"Serial must be provided") if serial.nil? or serial.to_s.empty? hash = @redis.hgetall("cert:#{serial}") if not hash.nil? and hash.has_key?("status") :status =&gt; hash["status"].to_i, :revocation_time =&gt; hash["revocation_time"].to_i || nil, :revocation_reason =&gt; hash["revocation_reason"].to_i || 0 ) else =&gt; R509::Validity::UNKNOWN) end end end end </code></pre> <p>If you just write tests for this, and your constructor looks like:</p> <pre><code> </code></pre> <p>then you're going to have to actually have a Redis database running for your tests, and you need to have the correct data in it in order for the tests to pass. Obviously, that sucks. You want to avoid that kind of environmental dependency in your unit tests.</p> <p>Enter mocking. Here's a pair of rspec tests that makes sure it returns an UNKNOWN status if nothing is found:</p> <pre><code>it "gets unknown when serial is not found (returns {})" do redis = double("redis") checker = redis.should_receive(:hgetall).with("cert:123").and_return({}) status = checker.check(123) status.status.should == R509::Validity::UNKNOWN end it "gets unknown when serial is not found (returns nil)" do redis = double("redis") checker = redis.should_receive(:hgetall).with("cert:123").and_return(nil) status = checker.check(123) status.status.should == R509::Validity::UNKNOWN end </code></pre> <p>(I have two tests here because I've seen the Redis driver return {} if a Hash isn't found, but I want to make sure my code still works if it returns nil.)</p> <p>The first key line is:</p> <pre><code>redis = double("redis") </code></pre> <p>It's called "double" because you're creating a "test double" object instead of an actual connection to the Redis database. The test double doesn't actually do anything, but if it receives a method call that you didn't tell it to expect (or if it doesn't receive a method call that you <em>did</em> tell it to expect), it'll give you a failing test.</p> <p>The next key line is where you tell your test double what to expect (and what it should do):</p> <pre><code>redis.should_receive(:hgetall).with("cert:123").and_return({}) </code></pre> <p>Let's break this down. You're telling your test double that it "should_receive" a call to the "hgetall" method (you use the symbol :hgetall for this), "with" the single parameter "cert:123", and it should return an empty hash object {}.</p> <p>The test double verifies that our Checker#check implementation does actually make this call (and only this call) to the Redis object, and it helpfully returns {}, which then exercises the "nothing found in the database" code path. Thus, we can then check that the returned status should equal R509::Validity::UNKNOWN (which would only happen if we reacted properly to the data we told the test double to return).</p> <p>And what about testing something else, for example that the serial number <em>does</em> exist in the database? Here's the spec for a certificate that's been revoked:</p> <pre><code>it "gets revoked with revocation time and reason" do redis = double("redis") checker = redis.should_receive(:hgetall).with("cert:123").and_return({"status" =&gt; "1", "revocation_time" =&gt; "789", "revocation_reason" =&gt; "5" }) status = checker.check(123) status.status.should == R509::Validity::REVOKED status.revocation_time.should == 789 status.revocation_reason.should == 5 end </code></pre> <p>Here, we say that our test double should receive the "hgetall" method with the single parameter "cert:123", and it'll return a populated hash with some certificate information (its status, revocation time, and revocation reason). We then verify that the status should be REVOKED, and that the revocation_time and revocation_reason are correctly translated to integers.</p> <p>Now, so far we've only seen expectations where the method should receive only one parameter. Expecting multiple parameters is exactly what you'd expect: just pass multiple parameters to should_receive. Here's an example of a spec for the R509::Validity::Redis::Writer class, where our test double will expect to receive multiple parameters:</p> <pre><code>it "when reason isn't provided" do redis = double("redis") writer = redis.should_receive(:hmset).with("cert:123", "status", 1, "revocation_time",, "revocation_reason", 0) writer.revoke(123) end </code></pre> <p>Here, our test double should receive the "hmset" method, with a slew of parameters (obviously, it maintains the type and order of the parameters, in addition to their values). Note also that we don't specify any "and_return" here, since we don't care about the return value from this method call.</p> <p>Hopefully that helps to explain rspec mocks, in the interest of getting you to make them yourself. It beats the hell out of having a database running for your unit tests to work.</p> Local Time Calculator for Java (Android) 2011-11-30T00:00:00-08:00 <p>Have you ever noticed that both MLB and NFL only give the starting times of their games in Eastern time? Through their websites, and even through their mobile apps -- which I find especially egregious since your phone knows what timezone it's in.</p> <p>One of the big features, in my opinion, that my <a href="">MLB Scoreboard</a> app has over MLB's official apps (or any other third party scoreboard apps that I know of) is that it converts the given Eastern times into your local time. I've just started using that same code in my new <a href="">NFL Scoreboard</a> app (which I've decided to make <a href="">open source</a>, for some reason).</p> <p>I want to be able to use this essentially like so:</p> <pre><code>LocalTimeCalculator calculator = new LocalTimeCalculator("12:30", "PM"); // if I'm in Central time (which I am), this will yield 11:30 AM Date localTime = calculator.getLocalTime().getTime(); </code></pre> <p>I want it to return a Calendar instead of a Date so I can easily set the date (year/month/day) if I want to; I do that sometimes, but I didn't feel that it should be part of the local time calculation.</p> <p>So here's my <a href="">LocalTimeCalculator</a>:</p> <pre><code>public class LocalTimeCalculator { private String mTime; private String mAmpm; public LocalTimeCalculator(String time, String ampm) { mTime = time; mAmpm = ampm; } public Calendar getLocalTime() { String[] timeArray = mTime.split(":"); int hour = Integer.parseInt(timeArray[0]); int minute = Integer.parseInt(timeArray[1]); hour = convertToHourOfDay(hour, mAmpm); Calendar eastern = Calendar.getInstance(TimeZone.getTimeZone("America/New_York")); eastern.set(Calendar.HOUR_OF_DAY, hour); eastern.set(Calendar.MINUTE, minute); Calendar local = Calendar.getInstance(); local.setTimeInMillis(eastern.getTimeInMillis()); return local; } private int convertToHourOfDay(int hour, String ampm) { if ("AM".equalsIgnoreCase(ampm)) { if (hour == 12) { return 0; } else { return hour; } } else { if (hour == 12) { return 12; } else { return hour + 12; } } } } </code></pre> <p>First, I split the "12:30" time into hour and minute, and then convert the hour to 24-hour time based on the AM/PM field. Then I get a Calendar instance set to Eastern time ("America/New_York"), and set its time. I get a new Calendar instance, which will default to local time,* and set the time on it based on the original Eastern Calendar.</p> <p><em>* On my Android phone, this updated as soon as I changed to a different timezone. It was working fine for me for the first few months of the MLB season before I finally went somewhere else (my trip to Las Vegas for DEFCON) and got to test that it was working in a timezone other than Central. Exactly what I was hoping for.</em></p> <p>I don't know if local time conversion was a big reason anybody wanted to use MLB Scoreboard or NFL Scoreboard rather than the official apps, but I do think it's the kind of thing all apps should do. Your phone knows what timezone it's in; you shouldn't have to keep translating times in your head.</p> <p>It's not complicated, and it makes things better for your users. If you're making Android apps, I think you should be doing this.</p> Android ListView: Maintain your scroll position when you refresh 2011-06-17T00:00:00-07:00 <p>Refreshing an Android ListView is a pretty common thing -- but the best way to do it isn't immediately obvious. Here's my progress through the different patterns.</p> <h2>The Obvious</h2> <p>I figured that when I've downloaded the new list of stuff I want to show, that I could just create a new adapter and stuff it into the list.</p> <pre><code>EventLogAdapter eventLogAdapter = new EventLogAdapter(mContext, events); mEventListView.setAdapter(eventLogAdapter); </code></pre> <p>This works, in that the list gets updated. But it always scrolls all the way back up to the top. Sometimes that might be what you want, but most of the time you'll want to maintain your scroll position.</p> <h2>The Naive</h2> <p>I tried getting pixel-level scroll position using getScrollY(), but it always returned 0. I don't know what it's supposed to do. I ended up going with a solution that got <em>close</em> to maintaining your scroll position.</p> <pre><code>int firstPosition = mEventListView.getFirstVisiblePosition(); EventLogAdapter eventLogAdapter = new EventLogAdapter(mContext, events); mEventListView.setAdapter(eventLogAdapter); mEventListView.setSelection(firstPosition); </code></pre> <p>This figures out the first item in the list you can see before resetting the adapter, and then scrolls you to it. This maintains your scroll position <em>within some unknown/arbitrary range</em>, and can cause you to jump around in the list a little bit when you refresh.</p> <p>If you're scrolled halfway through a list item, it'll snap you to the top of it so it's completely visible. Unfortunately, if you're scrolled halfway through a list item, that probably wasn't the one you were paying the closest attention to.</p> <h2>The Elegant</h2> <p>There had to be a better way!</p> <p>And, of course, there is. Romain Guy, an Android developer who haunts Stack Overflow and Google Groups dropping golden hints when people ask questions, <a href="">pointed out</a>:</p> <blockquote><p>The problem is that you are creating a new adapter every time you reload the data. That's not how you should use ListView and its adapter. Instead of setting a new adapter (which causes ListView to reset its state), simply update the content of the adapter already set on the ListView. And the selection/scroll position will be saved for you.</p></blockquote> <p>There are two problems with Romain Guy: 1) he doesn't have a central repository of these hints/answers so I can learn what to do before doing everything wrong first, and 2) they really are just <em>hints</em>, in that they point you in the right direction without getting you all the way there.</p> <p>In this case, yes, updating the adapter without creating a new one every time <em>will</em> maintain your scroll position. Except that you'll frequently get an "IllegalStateException: The content of the adapter has changed but ListView did not receive a notification. Make sure the content of your adapter is not modified from a background thread, but only from the UI thread."</p> <p>It turns out that you need to call notifyDataSetChanged() on your adapter after changing its contents; fortunately, that "only from the UI thread" bit was a red herring, because I didn't really want to do any processing that could/should be asynchronous on the UI thread.</p> <p>I added a refill() method to my adapters:</p> <pre><code>public void refill(List&lt;EventLog&gt; events) { mEvents.clear(); mEvents.addAll(events); notifyDataSetChanged(); } </code></pre> <p>And I call it when my download is complete:</p> <pre><code>if (mEventListView.getAdapter() == null) { EventLogAdapter eventLogAdapter = new EventLogAdapter(mContext, events); mEventListView.setAdapter(eventLogAdapter); } else { ((EventLogAdapter)mEventListView.getAdapter()).refill(events); } </code></pre> <p>If the list doesn't already have an adapter, then I create one. But if it <em>does</em> have an adapter, then I just refill it. And it maintains my scroll position exactly!</p> Simplified DAO helper for using JDO with Google Appengine 2011-02-09T00:00:00-08:00 <p>I've been playing with Appengine for Java lately, and using JDO (which stands for Java Data Objects) for the first time. All the example code is pretty annoying; apparently, in every DAO method, you have to grab a PersistenceManager from the PersistenceManager factory, use it to build and execute a Query, and then close the PersistenceManager before you return. The framework that has to be included in every method looks like this:</p> <pre><code>PersistenceManager pm = pmFactory.getPersistenceManager(); try { // do something with the pm here } finally { pm.close(); } </code></pre> <p>So that's five lines of crap, in every method. But the only thing that actually matters, that I actually want to write in my DAO methods, is the meaty part, the part that says "do something" in that comment there. So, I wrote a little library to let me do just that: it's my <a href="">vh-dao</a> project at Github.</p> <p>It starts with a DAO interface, which says that every DAO class has a store method; I've been on too many projects where sometimes that method is called "store" and sometimes it's "save" and sometimes it's "storeWidget" or "saveWidget" (where "Widget" is the name of the model class). Enough.</p> <pre><code>public interface VHDao&lt;T&gt; { public void store(T model); } </code></pre> <p>Then, I have an abstract base class that implements a few things that'll be the same for every DAO; including that store method, so when you create a new DAO you never have to worry about that.</p> <pre><code>public abstract class BaseVHDao&lt;T&gt; implements VHDao&lt;T&gt; { @Autowired protected PersistenceManagerFactory pmFactory; @Override public void store(T model) { PersistenceManager pm = pmFactory.getPersistenceManager(); try { pm.makePersistent(model); } finally { pm.close(); } } protected List&lt;T&gt; list(VHQuery vhQuery, Object... args) { PersistenceManager pm = pmFactory.getPersistenceManager(); try { Query query = pm.newQuery(vhQuery.getClazz()); if (vhQuery.hasFilter()) { query.setFilter(vhQuery.getFilter()); } if (vhQuery.hasOrdering()) { query.setOrdering(vhQuery.getOrdering()); } if (vhQuery.hasRange()) { query.setRange(vhQuery.getRangeStart(), vhQuery.getRangeEnd()); } List&lt;T&gt; list = (List&lt;T&gt;)query.executeWithArray(args); list.size(); return list; } finally { pm.close(); } } protected T first(VHQuery vhQuery, Object... args) { PersistenceManager pm = pmFactory.getPersistenceManager(); try { Query query = pm.newQuery(vhQuery.getClazz()); if (vhQuery.hasFilter()) { query.setFilter(vhQuery.getFilter()); } if (vhQuery.hasOrdering()) { query.setOrdering(vhQuery.getOrdering()); } query.setRange(0, 1); List&lt;T&gt; list = (List&lt;T&gt;)query.executeWithArray(args); if (list.size() &gt; 0) { return list.get(0); } else { return null; } } finally { pm.close(); } } } </code></pre> <p>As you can see, there are two methods that we can use in the subclass: list() and first(). They get a PersistenceManager, set up a Query based on the VHQuery you pass in, execute the Query based on all the (optional) parameters you give, and then close the PersistenceManager.</p> <p>That VHQuery object is there to define the filter/ordering/range that I need to enter into the actual JDO Query object; its reason for being is twofold:</p> <ul> <li>The JDO Query object is created by calling the PersistenceManager, which I won't have access to in the DAO subclass since I want to instantiate it in the base methods</li> <li>Prevent a leaky abstraction of requiring the subclass to know about the JDO Query object</li> </ul> <p>I originally had a method that built and returned the Query object that I could then fill in and execute, but I couldn't close the PersistenceManager after executing it, which would have led to annoying memory leaks. This seems nicer.</p> <p>Here's what a method looks like that just wants to grab a single item from the database:</p> <pre><code>public Greeting getLatestByAuthor(User author) { VHQuery query = new VHQuery(Greeting.class); query.setFilter("author == :author"); query.setOrdering("date desc"); return first(query, author); } </code></pre> <p>It just sets up the VHQuery and asks the base class to give it the first result that matches the given query parameters.</p> <p>And here's one that returns a list:</p> <pre><code>public List&lt;Greeting&gt; mostRecent(int count) { VHQuery query = new VHQuery(Greeting.class); query.setOrdering("date desc"); query.setRange(0, count); return list(query); } </code></pre> <p>As you can see, your DAO methods now have no extraneous lines that don't have anything to do with the query you're trying to execute. That's all handled generically for you, and you don't have to worry about it.</p> <p><strong>Note</strong>: I've written this so I can use it with the Appengine datastore, and haven't tried it with an SQL database. JDO is supposed to work just fine with an RDBMS, so this will probably help. I guess I'll find out when I get around to that.</p> <p>I'm guessing there was already something that does this, because I can't imagine that everyone using JDO is okay with typing all that boilerplate in every method; I couldn't find it, though, and am open to someone pointing me in the right direction. I'm also sure there will be more additions to this DAO abstraction as I find needs -- what do you think I've missed?</p> Using Ruby's extend and include to inject functionality into Sequel model classes 2011-02-02T00:00:00-08:00 <p>I was writing a program that needed to delete some items out of several different tables -- but we want to be safe and keep a record of what was deleted, in case we need to put it back in. Since we're guessing we won't find out what the problems will be, if any, until a lot of new data has been inserted into the database, simply dumping the database and reverting to a backup isn't going to be an adequate solution.</p> <p>I'm using Ruby here, along with the awesome Sequel ORM library.</p> <p>My idea is that whenever a record is deleted from our database, I want to create an identical record in a "slum" database. These slums won't be used for anything, hopefully, but if we need to use them they'll have everything -- including the id's -- that we deleted.</p> <p>At first, my models each had methods that supported this ... and it got pretty cumbersome. Then I remembered, "hey wait a minute, this is Ruby, and I can probably do something awesome!"</p> <p>Indeed, you can do awesome things.</p> <p>First, I created a "SlumBuilder" module that adds a class method called "build" to all the classes that <strong>extend</strong> it. In Ruby, when you <strong>extend</strong> a module its classes are included as class methods, and when you <strong>include</strong> a module its classes are included as instance methods. That's an important distinction, because we'll be doing both.</p> <p>Here's SlumBuilder:</p> <pre><code>module SlumBuilder def build(original) object = original.keys.each do |field| object[field] = original[field] end end end </code></pre> <p>As you can see, it instantiates a new object of whatever type has extended SlumBuilder, and fills it with all the fields (keys) in the given model object (original).</p> <p>A Slum model object might look like this:</p> <pre><code>class SlumCoolThing &lt; Sequel::Model(:cool_thing) extend SlumBuilder self.db = $slum_db end </code></pre> <p>That's it. It gets the available fields from the database (thank you Sequel!) and all its functionality from SlumBuilder. We just need to set the database it uses, since we're using two different databases.</p> <p>The regular model object, the one we'll need to delete some records out of, would look like this:</p> <pre><code>class CoolThing &lt; Sequel::Model(:cool_thing) include ModelDestroyer self.db = $regular_db end </code></pre> <p>Simple enough. As you can see, I went with the convention that the slum's model class is the same as the regular model's class with "Slum" prepended. This is important, as you'll see here in ModelDestroyer:</p> <pre><code>module ModelDestroyer def slum_class Object::const_get("Slum#{self.class}") end def destroy begin rescue # failed to build a slum, probably because the class doesn't exist, but we'll just continue on with the deletion puts "Failed to build slum: #{self.inspect}" end self.delete end end </code></pre> <p>This is where most of the magic happens. The "slum_class" method uses Ruby's cool "get the constant, like a class, that has the given name" functionality to get the class that has the same name as whichever one has included our ModelDestroyer with "Slum" prepended.</p> <p>Our "destroy" method gets that Slum class and calls its "build" class method, discussed above, that copies all the fields and saves a new record to the slum database. It then calls Sequel's "delete" method to remove the record from our database.</p> <p>So if we had a variable, cool_thing, that we pulled out of the CoolThing database and we wanted to delete it, we now have two options:</p> <pre><code>cool_thing.delete </code></pre> <p>Which just removes cool_thing from the database. The other option is:</p> <pre><code>cool_thing.destroy </code></pre> <p>Which removes cool_thing from the database <strong>AND</strong> inserts a perfect copy of it into a separate slum database.</p> <hr /> <p>You're probably not going to need to do exactly this, with inserting duplicate copies of a row upon deletion. But being able to inject functionality like SlumBuilder and ModelDestroyer into a class is really cool -- we couldn't use inheritance for this, because our model classes already inherit from Sequel::Model.</p> <p>It's bending my mind right now to think about how I'd do this in Java. I think it might be a bit more verbose ... if it's even possible.</p> Hibernate: Skipping Criteria's abstractions for HQL's flexibility and speed 2011-01-19T00:00:00-08:00 <p>I was writing a task that would run periodically and clear out old records from a database table, so the database doesn't grow unbounded; it's fortunate that we don't need to keep a perfect historical record in this table, and it sees a lot of inserts.</p> <p>We're using Java/MySQL/Hibernate here.</p> <p>My first pass used the standard "get them and delete them" pattern that seems so common when using Hibernate's Criteria API.</p> <pre><code>public List&lt;Record&gt; getByNameBeforeDate(String name, Date before) { return getHibernateTemplate().getSessionFactory().getCurrentSession() .createCriteria(Record.class) .add(Restrictions.eq("name", name)) .add("recordedAt", before)) .list(); } ... List&lt;Record&gt; recordsToDelete = recordDao.getByNameBeforeDate(name, date); for (Record record : recordsToDelete) { recordDao.delete(record); } </code></pre> <p>But I quickly became afraid of the way that would perform. I don't have enough data in my development environment to see what would happen when it runs, but it didn't take a whole lot of imagination to see it using a ton of memory, being too slow, and executing quite a few queries.</p> <p>I wanted something better. I couldn't figure a way around Criteria's limitations, so I went with HQL instead.</p> <pre><code>public int deleteByNameBeforeDate(String name, Date before) { String hql = "delete from Record where name = :name and recordedAt &lt; :before"; Query query = getHibernateTemplate().getSessionFactory().getCurrentSession().createQuery(hql); query.setString("name", name); query.setDate("before", before); int rowCount = query.executeUpdate(); return rowCount; } ... int count = recordDao.deleteByNameBeforeDate(name, date); </code></pre> <p>With the first method, memory use and queries executed were both O(N). With the second method, memory use should be close to constant (depending on what the database does), and queries executed will always be 1.</p> <p>So don't be afraid to ditch Criteria where it makes sense; HQL can be much more flexible in certain cases. It gets us much closer to what we'd do if we weren't using Java, which is just executing a single SQL query: all the fancy abstractions that Hibernate's Criteria API provides don't help with that.</p> <p>And remember to pay attention to the memory/speed characteristics of what you write. It can make a big difference.</p> Switching between different views in a Playbook app 2011-01-11T00:00:00-08:00 <p>In my Playbook apps, I often need to switch from one view to another. A common example is that I'm on my base screen that you see when you first launch the app, and you want to add or edit something in the list of items I'm showing you. I'll try to explain how I've done that.</p> <p>My add/edit screen is a subclass of View; in this case, I call it FamilyEditor.</p> <pre><code>&lt;?xml version="1.0" encoding="utf-8"?&gt; &lt;s:View xmlns:fx="" xmlns:s="library://" title="Family Editor" xmlns:mx="library://"&gt; </code></pre> <p>To switch from the base view to FamilyEditor, I need to call navigator.pushView(). Sometimes I want to add a new Family, and other times I want to edit an existing one. So I want to pass a Family to FamilyEditor. I do that by connecting these event handlers to my add/edit buttons:</p> <pre><code>private function addButtonClickHandler(event:Event):void { navigator.pushView(FamilyEditor, new Family()); } private function editButtonClickHandler(event:Event):void { navigator.pushView(FamilyEditor, familiesList.selectedItem); } </code></pre> <p>When you use MXML to extend a class, you don't get to define a constructor. So when you pass your argument into FamilyEditor, you need to override the data setter (just like you would when creating an item renderer).</p> <pre><code>public override function set data(value:Object):void { = value; family = Family(value); ... } </code></pre> <p>I can do my thing in the FamilyEditor view, and when I'm done I just call navigator.popView(). I do that when the user cancels, and when they click save I do it after performing the necessary saving operations. Here's the cancel button's event handler:</p> <pre><code>private function cancelButtonClickHandler(event:Event):void { navigator.popView(); } </code></pre> <p>That goes back to the view the user was on before they got here.</p> <p>So that's simple enough, I think. And you can do a whole lot with it.</p> Playbook/Flex: Delete button inside a List ItemRenderer 2010-12-22T00:00:00-08:00 <p>I'm doing a bit more Playbook development, and I wanted to be able to delete items out of a List with a button inside each item. The way to do that is by defining your own ItemRenderer.</p> <p>Here's what it looks like:</p> <p><img src="/wp-content/images/item_renderer_with_delete_button.png" alt="Item renderer with a delete button" /></p> <p>I'm using a ModelLocator just like the one I described in <a href="">my last post about Playbook development</a>, and I've defined a new MXML component that displays the name of the object and has a delete button in it.</p> <pre><code>&lt;?xml version="1.0" encoding="utf-8"?&gt; &lt;s:ItemRenderer xmlns:fx="" xmlns:s="library://" autoDrawBackground="true" xmlns:mx="library://"&gt; &lt;fx:Script&gt; &lt;![CDATA[ import model.ModelLocator; import model.Family; [Bindable] private var modelLocator:ModelLocator = ModelLocator.getInstance(); [Bindable] private var family:Family; public override function set data(value:Object):void { = value; family = Family(value); } private function deleteClickHandler(event:Event):void { modelLocator.removeFamily(family); } </code></pre> <p>And using the ItemRenderer (which I've named FamilyRenderer in this case, and placed in the views.renderer package where I've been putting all my renderers) is very simple.</p> <p>The ModelLocator.removeFamily method is about what you'd expect:</p> <pre><code>public function removeFamily(family:Family):void { var index:int = families.getItemIndex(family); families.removeItemAt(index); flush(); } </code></pre> <p>What happens here is that when you click that delete button, we call out to the ModelLocator and tell it to remove the item that's currently being rendered. Since the entire ModelLocator is [Bindable], the change to ModelLocator.persons will update what's displayed in the familiesList, and the ItemRenderer on which you just clicked the delete button disappears.]]></p> Snippets and patterns from my first Playbook app 2010-12-11T00:00:00-08:00 <p>This week I heard about RIM's plan to build a developer base for the upcoming Playbook tablet -- give a free one to everyone who builds an app and gets it accepted into their App World. Now, I don't know how stringent their review policy is, but the promise of a free tablet is enough to get me to write an app. I decided to port one of my iPhone apps, Actioneer, which has seemed constrained by the size of the phone's screen; hopefully an app that's good enough for Apple's App Store is good enough for RIM's App World.</p> <p>The app is done now. The basic concept of the app is that you can define a set of "Actions" that you'll perform from time to time; when you click "Perform" on one of them, it records the time the action was performed, and it will display a history of all the times each action was performed. Darlene uses it to help her remember how often Rusty craps, which apparently is valuable information for a mother.</p> <p>There are a few interesting things in the implementation of the app, that I want to highlight here. Playbook apps are written in Flex, and I've used the Cairngorm library pretty extensively in the past -- I had thought it was a little heavy, and I took this as an opportunity to take some ideas from it but make it a little bit lighter and easier to deal with.</p> <p>I start with the ModelLocator, which is a concept stolen from Cairngorm. It's a singleton that stores data that'll be shared between different views -- but I've added the concepts of persisting that data to local storage, as well as dispatching events. As I get around to larger apps, I'll probably find it necessary to split the persistence code out into another class, but I think the event dispatching works better here than in Cairngorm's FrontController.</p> <p>Here's my ActionModelLocator:</p> <pre><code>package model { import events.ActionAddedEvent; import; import; import; import mx.collections.ArrayCollection; [Bindable] public class ActionModelLocator extends EventDispatcher { private static var instance:ActionModelLocator; private var sharedObject:SharedObject; public var actions:ArrayCollection; public function addAction(action:Action):void { actions.addItem(action); dispatchEvent(new ActionAddedEvent(action)); flush(); } public function removeAction(action:Action):void { var index:int = actions.getItemIndex(action); if (index &gt;= 0) { actions.removeItemAt(index); flush(); } } private function flush():void { var serializedActions:ArrayCollection = new ArrayCollection(); for each (var action:Action in actions) { serializedActions.addItem(action.serialize()); } = serializedActions; sharedObject.flush(); } private function load():void { actions = new ArrayCollection(); if (sharedObject.size &gt; 0) { for each (var obj:Object in { actions.addItem(Action.deserialize(obj)); } } } public function ActionModelLocator() { if (instance != null) { throw new Error("Can only be one ActionModelLocator"); } sharedObject = SharedObject.getLocal("Actioneer_Actions"); load(); } public static function getInstance():ActionModelLocator { if (instance == null) { instance = new ActionModelLocator(); } return instance; } } } </code></pre> <p>The constructor, which can only be executed once, initializes the sharedObject variable by grabbing a local persistence object and calling load(), which will pull any data out of the local datastore and deserialize it for use. I'll get to the serialization/deserialization in a second, but first I want to finish explaining the ModelLocator.</p> <p>The converse of load() is flush(), which serializes all of our model objects and saves them to the SharedObject, and takes the extra step of flushing the SharedObject to disk. This last step probably isn't necessary -- the SharedObject is supposed to flush to disk when it needs to, like when the app is quit or event pushed from memory in the background. I don't trust that at the moment, though, so maybe once I actually have a Playbook I can investigate whether that works, and adjust this later.</p> <p>I have addAction/removeAction methods that handle adding and removing Action objects from the list and calling the aforementioned flush() method. The addAction method also has the added benefit of dispatching an ActionAddedEvent, because we'll want to be able to add an Action in one view and respond to an Action being added in another view.</p> <p>The events themselves are very simple; they extend the basic class, and all they need to do is define a name (which needs to be public/static and unique per VM) and encapsulate any objects that you want to pass along in the event. Here's the ActionAddedEvent:</p> <pre><code>package events { import; import model.Action; public class ActionAddedEvent extends Event { public static const NAME:String = "AddActionEvent"; public var action:Action; public function ActionAddedEvent(action:Action, type:String=NAME, bubbles:Boolean=false, cancelable:Boolean=false) { super(type, bubbles, cancelable); this.action = action; } } } </code></pre> <p>To respond to an event, you just need to add an event listener to the model locator that will dispatch the events. I did mine in the creationComplete handler of my views:</p> <pre><code>private function onCreationComplete(event:Event):void { actionModelLocator.addEventListener(ActionAddedEvent.NAME, function(e:ActionAddedEvent):void { ... }); } </code></pre> <p>Finally, I'll show you the Action model object, which holds the data for each action and handles serialization/deserialization of each action.</p> <pre><code>package model { import mx.collections.ArrayCollection; [Bindable] public class Action { public var name:String; public var history:ArrayCollection; public function Action(name:String=null, history:ArrayCollection=null) { = name; if (history == null) { history = new ArrayCollection(); } this.history = history; } public function perform():void { history.addItemAt(new Date(), 0); } public function deleteHistory(date:Date):void { var index:int = history.getItemIndex(date); if (index &gt;= 0) { history.removeItemAt(index); } } public function serialize():Object { var obj:Object = { name:, history: this.history }; return obj; } public static function deserialize(obj:Object):Action { return new Action(, obj.history); } } } </code></pre> <p>I'd heard a rumor that when you save something to SharedObject, it would serialize ActionScript objects for you, so you'd be able to put them in and pull them out easily. No such luck, however. When I tried to save an Action directly into the SharedObject, it came back out as a generic Object that couldn't be cast or coerced into an Action -- it was stuck. So my serialize/deserialize methods convert the Action into and out of an Object that the SharedObject persistence can use. (Note: the serialize() method may not be totally necessary in this case, but I'm using it to demonstrate a pattern I expect to have to use in general.)</p> <p>Hopefully that gives you some ideas as you build your own Playbook apps. I'm not thrilled with the level of documentation, and the FlashBuilder IDE keeps steering me away from components I would normally use that aren't "mobile optimized," so I have a lot more to learn when it comes to working with MXML in my Playbook apps.</p> <p>And I'm curious to see how much of this stuff I can pull out into libraries as I build my next Playbook apps.</p> Random Gift Assignment 2010-11-24T00:00:00-08:00 <p>This year, my family is thinking about changing the way they exchange gifts between each other. The basic idea is that rather than having every single person give a gift to every other person, each person would only give a gift to one other person. Saves money, makes things easier, seems like a positive all around. But how to choose who gives something to whom?</p> <p>Enter the Gifter program. It takes a list of names, assigns each one as giving a gift to one other, and outputs the assignments. Each name only appears once as a gift-receiver, and it doesn't allow any name to be assigned to itself -- it kind of defeats the purpose if you get to give a gift to yourself, doesn't it?</p> <pre><code>class Gifter def initialize(names) @names = names end def unused_names(all_names, used_names) all_names - used_names end def random_name(names) names[rand(names.size)] end def random_name_excluding(names, exclude) if names.size == 1 and names[0] == exclude raise "Cannot assign #{exclude} to self" end name = self.random_name(names) while name == exclude name = self.random_name(names) end name end def try_to_give gifts = {} @names.each do |name| gifts[name] = self.random_name_excluding(self.unused_names(@names, gifts.values), name) end gifts end def give if @names.size == 1 raise "You can't exchange gifts if you're all by yourself, you poor lonely fool!" end loop do begin return try_to_give rescue end end end end names ="names").split("\n") gifter = gifts = gifter.give gifts.keys.each do |giver| puts "#{giver}: #{gifts[giver]}" end </code></pre> <p>It reads the names out of a file, expecting that each name is on its own line. Then it attempts to assign each name to one other name, and if it gets to the end and finds that its only option is to assign a name to itself, it throws away all the assignments and tries again. That can happen a few times, but the likelihood of it happening repeatedly, to the point where you'd notice the program running slowly, is pretty astronomical. So I'm not worried about it.</p> <p>We'll see if we end up doing it this way. But if you want to try it with your family (or group of friends), I guarantee this program is easier than trying to come up with a fair and random assignment by hand.</p> Killing Tomcat, the fun and easy way! 2010-11-06T00:00:00-07:00 <p>For the longest time, I've gone through an annoying manual step of killing Tomcat when it needs to die.</p> <pre><code>ps aux | grep tomcat </code></pre> <p><em>Search for the right one (ie, not the grep), and manually enter the process id into:</em></p> <pre><code>kill -9 &lt;process id&gt; </code></pre> <p>Normally, Tedium is a good enough motivator for me to script something like this. But this time, I waited until Ridicule threw its hat into the ring. After a coworker asked why I didn't script it, it was really only a matter of time.</p> <pre><code>#!/usr/bin/env ruby ps = `ps aux | grep catalina.startup.Bootstrap | grep -v grep` chunks = ps.split(" ") if not chunks.empty? pid = chunks[1] puts "Killing pid: #{pid}" `kill -9 #{pid}` end </code></pre> <p>And now all I have to do is:</p> <pre><code>~/code/killer/kill_tomcat.rb </code></pre> <p>There you go. Tomcat is dead.</p> Increasing screen brightness with the /proc filesystem 2010-10-17T00:00:00-07:00 <p>I installed Fedora/LXDE on an old laptop this weekend, and it went well -- but when I told it to hibernate and then booted it back up, the screen was painfully dim. I wonder why that happened ... but since I'm not planning to go fixing any bugs in the hibernate code of either Fedora or LXDE, the <em>why it happened</em> wasn't nearly as pressing a question as <em>what do you do about it</em>?</p> <p>So I started digging around in the /proc filesystem, after not finding anything obviously related to screen brightness in 5 seconds of looking through the GUI tools. The <strong>/proc/acpi/video/VGA/LCDD/brightness</strong> file looked promising.</p> <pre><code>[root@flower sirsean]# cat /proc/acpi/video/VGA/LCDD/brightness levels: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 current: 0 </code></pre> <p>Ah ha, no wonder the screen was so dim! I just had to tell it to increase the brightness.</p> <pre><code>[root@flower sirsean]# echo 5 &gt; /proc/acpi/video/VGA/LCDD/brightness </code></pre> <p>The screen <em>immediately</em> got much brighter, more usable. I was kind of surprised by that, to be honest, but I liked it.</p> <pre><code>[root@flower sirsean]# cat /proc/acpi/video/VGA/LCDD/brightness levels: 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 current: 5 </code></pre> <p>Yup. I have a feeling I'll be needing to do this again at some point.</p> 3D Ultrasound of Rusty 2010-07-11T00:00:00-07:00 <p>So we went to have some 3D ultrasound images taken this weekend, which I'd never heard of before. The idea of it, though, is pretty cool. I don't know if you've ever seen an ultrasound happen, but the basic concept is that they take a slice image that you can see; sure, it lets you see more than you would without the ultrasound at all, but it also leaves quite a bit to the imagination. Some people can't tell what's going on at all.</p> <p>But if you take those slices and let a computer reconstruct them into a 3D image, it looks much better. You'll probably be able to tell what you're looking at. Watch this:</p> <p><video src="" width="624" height="460" controls preload></video></p> <p>(This is an HTML video in H.264 format. Use Safari or Chrome.)</p> Chrome OS: Remapping the Caps Lock key, plus some initial thoughts 2010-05-16T00:00:00-07:00 <p>This morning I installed Chromium OS on my old Acer Aspire One, which happens to be on its last legs and has been collecting dust in the bottom of a box in the back of my closet for the last several months. (The biggest two problems are 1: the AC adapter doesn't work consistently and sometimes won't charge the device, and 2: the OS options are slim, and slow, and Jolicloud was a huge disappointment.)</p> <p>I used <a href="">the Chromium Flow build by Hexxeh</a>, which is just a 300MB or so download and gives you a bootable USB drive you can use to check out Chromium OS -- or install to the computer's hard drive so you can use it without having a USB drive sticking out the side of your computer.</p> <p>So far the main problem I've seen is that this thing supports Flash; that makes web pages load <em>painfully</em> slow on a device with merely a 1.6 GHz Atom and 512 MB RAM. If Google wants Chrome OS to be a success, they're going to have to <em>really</em> increase the minimum hardware requirements, or implement a very high quality version of "click to Flash" or something which blocks Flash from loading until the user explicitly asks for it. But this isn't about Flash, so I'll just leave it at that. Other than Flash, the Chromium OS experience is quite good.</p> <p>With the tiny keyboard on the netbook, I was having quite a bit of trouble hitting the control keys; they're really small and tucked away down in the corner of the keyboard where my oversized, gnarled up fingers can't reach. So I hit <strong>Ctrl-Alt-T</strong> to open up a terminal window, which includes a cool window-sliding animation, and entered the following commands:</p> <pre><code>xmodmap -e "remove lock = Caps_Lock" xmodmap -e "add control = Caps_Lock" </code></pre> <p>Then I switched back to the main browser window by hitting <strong>Alt-Tab</strong>, though hitting <strong>F12</strong> would give me an Expose-like panel view of all my open windows.</p> <p><strong>Note:</strong> The only way I've figured out to close terminal windows once they're open is to go back into them and issue the "exit" command. Chromium OS doesn't seem to have very robust window-management functionality, which isn't surprising given the way they'd expect you to use it.</p> <p>So now the Caps Lock key is a Control button, and using the keyboard just got a whole lot more pleasant.</p> <p>With some sexier hardware, a more comfortable keyboard, and a 3G connection, I can totally see myself using a Chrome OS device; that said, I can't see how they can ever make it as nice to use as an iPad.</p> Simple Javascript event/model framework 2010-04-25T00:00:00-07:00 <p>Last weekend I wrote my first iPhone app, Tabata Sprinter, and Apple let it into their App Store late in the week. I took that as validation that using Appcelerator Titanium to write iPhone apps is still acceptable, because that's what I'd used. So this weekend, I wrote a second app, Choose4me, which is currently awaiting review.</p> <p>While making the first app I started a model/event binding library, and I have improved upon it for the second app.</p> <p>The concept is that you want to store your values in a central location, and listen for any changes in their values so you can update the UI or fire any other events that you want. Without further ado, here's the code:</p> <pre><code>var VH = { Model: function () { this._listeners = {}; this._fields = {}; this._boundListeners = {}; this.listen = function(key, listener) { if (this._listeners[key] == null) { this._listeners[key] = []; } this._listeners[key].push(listener); }; this.set = function(key, value) { var oldValue = this._fields[key]; this._fields[key] = value; if (oldValue !== value) { this.poke(key); } }; this.poke = function(key) { if (this._listeners[key] != null) { for (var i=0; i &lt; this._listeners[key].length; i++) { var listener = this._listeners[key][i]; listener(this._fields[key]); } } }; this.get = function(key) { return this._fields[key]; }; } }; </code></pre> <p>To listen for a changing variable, you call listen() and pass it a callback that takes a single argument -- the newly changed value.</p> <p>When you set() a variable, it checks to see if the new value is different from the old value, and if so, notifies all listeners using poke(). Note that you can call poke() yourself, if you want to notify listeners without setting a value (for events, perhaps).</p> <h3>Example</h3> <p>You instantiate your model at the beginning; it should be among the first things you do.</p> <pre><code>var model = new VH.Model(); var delegate = new VH.Model(); </code></pre> <p>I'm using two of them; the "model" is for storing values, and the "delegate" is just for firing events (ie, you use the key/value pairs but you don't care about the values). This is just a convention I came up with, you may not find it useful.</p> <p>Then you set default values, which your components may need during their setup.</p> <pre><code>model.set(C4.fields.numOptions, 2); </code></pre> <p>(Note that C4.fields.numOptions is just a string constant. You'll want each key to be a unique value, and using constants helps keep track.)</p> <p>While you're setting up your components, you'll want to listen for changing values.</p> <pre><code>delegate.listen(, function(v) { model.set(C4.fields.currentChoice, VH.random(1, model.get(C4.fields.numOptions))); }); </code></pre> <p>Here, we're listening for any time someone calls delegate.poke(, which indicates that the "choose" event has been fired. So when does that happen?</p> <pre><code>chooseButton.addEventListener("click", function(e) { delegate.poke(; }); </code></pre> <p>I have a button, and I attached an event listener to it, so that any time they click the button we'll poke the delegate to fire the choose event. I could have done this differently -- either by doing the random calculation right in the click event handler and setting the model field directly, or by using a function rather than listening for the choose event. But this works better because we get the benefit of writing our logic only once just like a function, while avoiding any "function is called before it's defined" warnings that crop up during cross-compilation.</p> <p>Using this model has worked out great for me in my first two apps, and what's nice is that I expect it'll also work nicely in a regular webapp the next time I write one.</p> <p>So hopefully Appcelerator Titanium remains kosher on the App Store, because this is a pretty fun way to program.</p> Using function scope to build a jQuery event handler 2010-03-04T00:00:00-08:00 <p>So here's another cool thing from the development of <a href=""></a> that might interest some people.</p> <p>When I'm creating or editing polls, I have the option of adding an arbitrary number of answers. I don't want to just leave an infinite number of form fields for the user to potentially fill in if he wants to -- that's as stupid as it is impossible. So they have to be able to take an action that adds a field dynamically.</p> <p>The first time I did it, on the poll creation screen, it was just a feature; I didn't need to abstract it, so I didn't. But once I needed it on the editing screen too, it was time to abstract. So here's what it looks like.</p> <p>First the creation screen, where there are three default answer fields and the ability to add more:</p> <pre><code>%ol#answers_list %li %input.answer{ :type =&gt; "text", :name =&gt; "answers[]", :tabindex =&gt; 2 } %li %input.answer{ :type =&gt; "text", :name =&gt; "answers[]", :tabindex =&gt; 3 } %li %input.answer{ :type =&gt; "text", :name =&gt; "answers[]", :tabindex =&gt; 4 } %p %a#add_answer_link{ :href =&gt; "#" }="add another answer" </code></pre> <p> And now the link needs to be connected to its functionality:</p> <pre><code> :javascript $(document).ready(function() { $("#add_answer_link").click(add_answer_callback(3, 5, "#answers_list")); }); </code></pre> <p>I'm just adding a click event handler to the link, and it's the result of the add_answer_callback() function.</p> <p>Now, the editing screen, where form fields only exist if they're filled in. I'm keeping track of the proper tabindex with a variable here ... it seems dirty to me so if anyone knows enough about Haml to do this more elegantly I'd like to hear about it.</p> <pre><code>%ol#answers_list - tabindex = 2 - @answers.each do |answer| %li %input.answer{ :type =&gt; "text", :name =&gt; "answers[]", :tabindex =&gt; "#{tabindex}", :value =&gt; "#{answer}" } - tabindex += 1 %p %a#add_answer_link{ :href =&gt; "#" }="add another answer" </code></pre> <p>And then connecting the event handler is pretty much exactly the same:</p> <pre><code>:javascript $(document).ready(function() { $("#add_answer_link").click(add_answer_callback(#{@answers.count}, #{@answers.count + 2}, "#answers_list")); }); </code></pre> <p>Note that instead of using constants for the parameters, I'm using calculated values. You can probably tell from here, but the parameters I'm passing in are: "the 0-based index of the next answer," "the 1-based tabindex of the next form field," and "the jQuery selector to which to append the field." Maybe that wasn't as obvious as I thought. It's a good thing the code has comments (in the source, that is, I'm not bothering with them here).</p> <p>So I'm calling a function to get the event handler, but the event handler itself is supposed to be a function. This is Javascript now, so that's not hard in the slightest.</p> <pre><code>function add_answer_callback(answer_index, tabindex, append_to) { return function() { $('&lt;li&gt;&lt;input id="answer_' + answer_index + '" type="text" class="answer" name="answers[]" tabindex="' + tabindex + '" /&gt;&lt;/li&gt;').appendTo(append_to); $('#answer_' + answer_index).focus(); answer_index++; tabindex++; } } </code></pre> <p>This function takes its three parameters and returns a function that takes <em>no</em> parameters and can be used as an event handler. Nice. But the <em>really cool</em> thing about it is that the answer_index and tabindex variables are incremented inside the event handler (ie, that happens each time you click the link), but since they were parameters on the outer function, they're scoped within the add_answer_callback() function, which is only called once (when the page is loaded).</p> <p>That way, the id and tabindex of each field are set properly as you add more fields, and because you're passing them in it doesn't even matter how many you started with (crucial for the editing screen, of course).</p> Authenticating with MongoMapper 2010-03-04T00:00:00-08:00 <p>For my new <a href=""></a> application, I'm using MongoDB and need to be able to connect to the authenticated database in production, but not necessarily in my development environment. I couldn't find anywhere online that shows specifically how to do that, so I'll post it here.</p> <pre><code>MongoMapper.connection =['db_hostname']) MongoMapper.database = config['db_name'] if config['db_username'] MongoMapper.connection[config['db_name']].authenticate(config['db_username'], config['db_password']) end </code></pre> <p>Note that I'm using MongoMapper, which is excellent.</p> <p>The key here is that I set up a connection using the hostname (or IP address) of the database server, then set the database I want to use. Then <em>if I've specified authentication parameters</em>, it attempts to authenticate against the database server. If I haven't specified authentication information, this will assume I want to try to connect unauthenticated (and my database is running in non-authenticated mode).</p> <p>Pretty simple.</p> Twitter tells the story of the great Hawaii tsunami of 2010 2010-02-28T00:00:00-08:00 <p>Twas February 26, and it'd been a long day. I'd spent most of it in the sun, as it was my first full day in Hawaii. At 8:30 PM local time I dozed off, visions of sugar plums dancing, for whatever reason, in my head.</p> <p>Minutes later, 6500 miles away, an earthquake rent the earth in Chile; it rated 8.8 on the Richter scale, a very large quake on the same fault line as the largest ever (9.5), which occurred in 1960. That one caused a tsunami in the Pacific Ocean which ended up killing 61 people in Hawaii. Tons and tons of water raced from the South American coast, at 500 miles per hour or more, speeding towards me as I slept.</p> <p>I had no idea, of course, that this was happening. I woke up in the middle of the night, about 2 AM or so, as I'd done the previous night. I turned off the air conditioner and listened to the rain; on Kauai, like Camelot, it apparently only rains at night. I easily fell back asleep, with no knowledge of what was coming.</p> <p>Before I knew it, I was roused from my sleep by the phone ringing. My mom's name was prominently displayed on the screen, and I answered. I quickly found out that she thought it was early in the morning; I checked, and it was 5:40 AM. She was apologetic that she'd woken me up, and wasn't getting to the point. Like a mother, she was trying to soften the blow. I heard my little sister's voice in the background: "Mom, just tell him!" She told me that there'd been an earthquake in Chile, which I thought was pretty bad news and I hoped things were okay there ... it hadn't been that long since the devastation in Haiti.</p> <p>Once again, I heard my sister's voice. "Mom, tell him!" And that's when my mom got to the point: <em>"There's a tsunami warning in Hawaii, right now. They're sounding the alarm at 6 AM."</em> I thanked her and got off the phone as quickly as I could. I sent Darlene to the shower and checked Twitter to confirm whether this was happening; since "Chile" and "Hawaii" and "Tsunami" were all trending, I turned on the TV to find out what was going on; at the time, less than 120 deaths had been reported in Chile, and the local emergency services were warning me to "stay tuned." I looked up the Kauai evacuation plans online, but they seemed outdated and incomplete. Also, it's 2010. Can your map please not be a JPEG of black outlines on a white background with few landmarks and no interactivity?</p> <p><a href="">sirsean</a>: <strong>This news about a tsunami hitting Hawaii sucks. Hopefully we don't get killed.</strong></p> <p>I threw my laptop and some charger cables in my bag, and was about to take a shower of my own when the clock struck six and the siren sounded -- it was time, as I once heard someone say, to get the h out of d. We left our luggage in the room and went to the lobby, carrying only a Macbook, a couple iPhones, a camera, and a bottle of water. In the lobby Darlene bought a couple handfuls of snacks and we unsuccessfully tried to call a cab; evidently, they were all already busy.</p> <p><a href="">sirsean</a>: <strong>Still don't know anything. Waiting for evac info. #Hawaii #tsunami</strong></p> <p>The woman behind the desk explained that we were supposed to evacuate to the nearby Kapaa High School, and gave us some very rough directions: "Get to the main highway and turn right." We tagged along with a nervous couple who had a rental car, and we set off to high ground; there was another couple stuffed in the back seat with us, but comfort was the last thing on any of our minds (or at least it was the last thing on my mind ... if I'm wrong about the thoughts of the other people in the back seat with me, well, oops).</p> <p>The guy in the passenger seat pulled out his Garmin GPS device to find where we were supposed to go. Meanwhile, my iPhone was doing the same thing. When the Garmin (which apparently takes "a few minutes" to come up with a route, compared to "a few seconds" on the iPhone) chirped "turn left!" my head jerked up. The iPhone was saying we needed to go straight for another mile or so. I brought it up, but our ride trusted the Garmin.</p> <p>It didn't seem like a big deal, because we were going up a hill and after a few minutes we found ourselves at what looked like a school. This could be Kapaa High School, right?</p> <p><a href="">sirsean</a>: <strong>We got to a "safe ground" up at a middle school evac zone. About 50-60 feet above sea level, I'd say. Maybe more.</strong></p> <p>Nope. A police officer in the parking lot informed us that it was a middle school; the doors were locked and since it was a state-funded school, the county-funded police couldn't get in. We, and the tiny handful of other people who had mistakenly found our way here, had to wait for someone from the school to show up, on a Saturday, to open it up.</p> <p>The people who took us there left at this point, to go onward to Princeville hotel. For some reason they were only staying out our hotel for one night, and then at Princeville for the rest of their vacation. I'd heard from my boss that the Princeville hotel offers a delicious brunch, and considered going with them to try it out. But I decided to stay at the school; perhaps moreso, I decided I didn't want to try to explain to them why I wanted to keep traveling in their car, didn't want to explain to everyone else why my only reason for it was for brunch, and didn't want to figure out how to get back afterwards.</p> <p><a href="">sirsean</a>: <strong>Phone calls aren't going through at least I can get Internet access. (Mom &amp; Dad, I was just about to call you to say I was ok. Don't worry.)</strong></p> <p>I assume a lot of people were trying to use their phones. Since my iPhone uses AT&amp;T, mine is basically useless for that purpose; whenever I tried to make a call that day, it failed to connect. But I had five bars everywhere, and the internet remained fairly responsive. And frankly, I'd much rather have access to the internet than be able to make a phone call.</p> <p><a href="">sirsean</a>: <strong>Yesterday's sunrise was more pleasant than today's. Just saying. (Damn you, nature!)</strong></p> <p>It'd been dark when we left the hotel, and in the confusion of figuring out where we were, the sun had snuck above the horizon without my noticing. At the time, this annoyed me.</p> <p><a href="">sirsean</a>: <strong>At least it's still scenic around here.</strong></p> <p><img src="" alt="Scenic" /></p> <p>By this point, many of my friends had remembered that I was in Hawaii and had wished me good luck and advised me to stay safe, via Twitter mentions. A coworker, Paul, says it best.</p> <p><a href="">reaperhulk</a>: <strong>@sirsean Twitter has become an essential service for information dissemination to/from those in affected areas. Amazing/crazy/scary.</strong></p> <p>I couldn't agree more. From where I stood, Twitter was by far the most reliable communication medium available during this crisis; it sure beat the phone system. I suppose email would have been as reliable, but for a few things:</p> <ul> <li>Who do you decide to send a message to, when so many people may be interested?</li> <li>Similarly, what if someone else with valuable information <em>doesn't</em> send something to you when you might need to know it?</li> <li>My email inbox was quickly swamped by "you have a new follower" emails from Twitter (not yet, of course, but soon). The information density of email is very low, like a mother trying not to frighten you.</li> </ul> <p><a href="">sirsean</a>: <strong>Thanks to everyone for the well-wishes! My battery is still close to full, &amp; I'm glad for Twitter.</strong></p> <p>Other people were worried about my battery life, and so was I. I needed to make sure to find some electricity at some point, because I didn't know how long the power would last once the tsunami arrived.</p> <p>The people from the school still hadn't arrived, but the grounds were pretty wide open; when building this school they took advantage of the fact that it's on Kauai, and replaced all the hallways that normally exist in a school building with open walkways. Every classroom was its own building. So we took it upon ourselves to wander around.</p> <p><a href="">sirsean</a>: <strong>Get ready for this view. That's the direction the tsunami will be coming from. I'm going to try to get some footage</strong></p> <p><img src="" alt="Upcoming view" /></p> <p>Looking at that picture now, it's tough to see the water. That's disappointing, because at the time I felt that I could see quite a bit of the water. I suppose it's tough for the iPhone camera to take pictures with the sun in the foreground.</p> <p>The first person exhorting others to follow me for a first-hand account of the coming tsunami was a former coworker, Ray:</p> <p><a href="">raykrueger</a>: <strong>My friend @sirsean is in Hawaii right now waiting for more Tsunami news/instructions. I imagine he'll be posting some photos too.</strong></p> <p>I think the picture of the view I'd posted didn't instill much confidence -- people were telling me to get to high ground. The police had insisted that we were high enough already and that they didn't want anyone on the roads.</p> <p><a href="">sirsean</a>: <strong>We're at the highest ground we're going to get to, so now we play the waiting game.</strong></p> <p>They repeatedly said "We wouldn't leave all our police cars and county vehicles here if it wasn't safe." Frankly, I thought that was cute. But it's a pretty good way to keep people calm, at the maximum possible expense of having to buy new cars (and likely at no cost at all).</p> <p><a href="">sirsean</a>: <strong>This is a middle school? I'd love to have attended. But being a refugee isn't as fun as being a student.</strong></p> <p><img src="" alt="Kapaa Middle School" /></p> <p>Seriously, the school looked great. I later learned that it was built in 1997, and that while it's designed for 1200 students (grades 6-8), it currently only has 650. Also, the principal said that it's a fine shelter during a tsunami, but when they have hurricane warnings it's a terrible place to go because the architects wanted "lots of big glass windows" and "light" -- the only place to be safe from a hurricane was to lock yourself into the one building with no windows (there are dozens of buildings, and even the gym has glass and few walls). It was a good point, but I just found myself glad that the architects liked "light" so much and that there wasn't a hurricane coming. I ended up never going into that windowless building, but the principal didn't make it sound pretty.</p> <p>Speaking of the gym, they took us there now. By this point, about 500 people had shown up here. Officer DeBlake, the same one who had earlier assured us that we'd be safe here, briefed us. He explained that we'd done the right thing by leaving the coastal areas, but that this isn't an official shelter so there'd be no Red Cross support, or medical assistance, or food. He said we were a low priority for the police and other rescue forces -- the people who had not evacuated and were still down by the coast were the highest priorities, followed by those at official shelters. He explained that right after he was done talking, all but one police officer were going to leave and go down the mountain to help others.</p> <p><a href="">sirsean</a>: <strong>We were just briefed. They said we should plan to be here 24+ hrs, that all the cops are leaving to help others, and that we're low priority</strong></p> <p>Oh yeah, and he said we had to start planning to be there for 24 hours. My hotel was right on the beach, so if the tsunami actually ended up as anything to worry about, the hotel would be pretty wrecked up anyway. But I didn't like hearing about being stuck in a shelter for 24 hours with 500 other people and no food. Darlene and I had already eaten a little bag of Doritos and a peanut bar of some kind. That was about a quarter of our foor supply, so at that point I figured we had to make 500 empty calories last another day between the two of us. It was time to start rationing, so I had to tell Darlene "no" when she asked if she could have some food (which was funny, I thought, because she was carrying it and didn't need me to say "yes"). Still, I knew that she'd be the one to eat all that food.</p> <p><a href="">sirsean</a>: <strong>I know of aftershocks of 6.9 &amp; 6.6 within 2 hours of the 1st quake. Also, they don't have many resources to help people here.</strong></p> <p>Given those aftershocks, I thought maybe the initial wave wouldn't be the end of it, and we'd see some big waves for several hours after it started. Also, office DeBlake had mentioned that Kapaa had only one fire station and one ambulance for the entire town. That, I thought, gave some idea about the level of resources available in town.</p> <p><a href="">sirsean</a>: <strong>Found an outlet, so I'm charging up before the power goes out.</strong></p> <p>I was already down to 80% after all that tweeting, so I was happy to find outlets available <em>outside</em> the buildings. They thought of everything at this school.</p> <p><a href="">sirsean</a>: <strong>They got us a TV. Should calm people down. I asked them if they need help, but they don't seem to think they do. They'll change their mind.</strong></p> <p>That was a bit of an exaggeration. When I offered my help, the principal asked me to "turn off the radio," which was 15 feet away and had a prominent "off" button waiting to be pressed. (It didn't seem like I was being all that helpful.) And here's a spoiler alert: they didn't change their minds. I don't know if they would have if things had gotten bad. Maybe.</p> <p>My cousin, Darrell, was the next to offer up my Twitter feed to those interested.</p> <p><a href="">darrell_schulte</a>: <strong>Follow Minnesota native @sirsean for your Hawaii tsunami coverage. If he drops a F bomb, it's not my fault.</strong></p> <p>I generally don't mind swearing, but this made me want to watch my mouth for the rest of the day. You never know how people are going to react when their sensibilities are offended <em>and</em> there's a huge natural disaster that costs lives. That's a pretty dangerous combination. (Though I'd contend that "a huge natural disaster that costs lives" combined with <em>anything</em> is a pretty dangerous combination. Whatever.)</p> <p><a href="">sirsean</a>: <strong>Filled up my water bottle; they're shutting off non-emergency water supplies in 17 minutes. #hawaii</strong></p> <p>I learned from the television that they were shutting off the non-emergency water sources at 10 AM, so it was time to get as much water right now as possible. Since we weren't at an official shelter, I didn't know if they'd consider our fountains an "emergency" water source.</p> <p>It shocked me, by the way, how often I had to walk over to someone in the ensuing several minutes and tell them to fill up their water bottles. I don't know what they were paying attention to, but it wasn't the urgent information about their survival. One woman even argued with me about it; I'd interrupted her reading a book with an empty water bottle next to her, and after telling her that they might be shutting off the water soon she said "I didn't hear that." Her back was turned to the TV and she'd been pretty engaged in the book -- I didn't see how her failing to hear something meant it hadn't been said. I thought, for a moment, about what I could say to her. "Did you hear anything?" "Does it matter? You're going to want water anyway." "Better safe than sorry." But I didn't think any of those would get through to her. So: "They said it 30 minutes ago." It worked. She got up and went to refill her water.</p> <p><a href="">sirsean</a>: <strong>Just learned there have been 56 aftershocks of more than 5.0</strong></p> <p>That seemed like a lot of earthquakes. There didn't seem to be any reason to doubt that there'd be some crazy waves coming soon.</p> <p><a href="">anoopbhat</a>: <strong>Wow. @sirsean I forgot you were in HI. Dude be careful and take photos. We're glued to your stream now.</strong></p> <p>Another coworker, Anoop, told me he was now paying attention. I don't know what it's like for people who often have an audience, but having people "glued to [my] stream" was pretty novel and I thought it was cool. At this point, I was starting to feel some responsibility to report accurately and often.</p> <p><a href="">sirsean</a>: <strong>TV is warning: "Never surf a tsunami." I'm stunned they have to say that. Maybe I shouldn't be.</strong></p> <p>Of course, that doesn't mean there shouldn't be any levity involved.</p> <p><a href="">sirsean</a>: <strong>We're 1 hour from some action here. #hawaii</strong></p> <p>Estimated action, that is. The scientists the newsmedia were talking to estimated that the first wave would arrive at 11:05 AM.</p> <p><a href="">sirsean</a>: <strong>Just saw TV footage of a lone surfer still out there, ignoring coast guard &amp; fire department helicopters.</strong></p> <p>I'm not going to lie, he looked like he was having fun.</p> <p><a href="">sirsean</a>: <strong>TV meteorologist: "The surfing is great right now. Make no mistake about that." This seems like a fun state.</strong></p> <p>Yeah, the reporters seemed to agree with me. But saying stuff like that probably didn't discourage other surfers from hitting the water.</p> <p><a href="">sirsean</a>: <strong>We're becoming an official shelter in a few minutes. Getting more refugees, and some more people to manage them. Good news.</strong></p> <p>Apparently they were moving some people from the nearby high school (the one we were supposed to go to) and bringing them here. I didn't know why, but if it meant the Red Cross was going to show up with food, I was all for it.</p> <p><a href="">sirsean</a>: <strong>Just found out a friend of mine is hiking in Chile right now. I hope she's alright!</strong></p> <p>My sister texted me to tell me about that. Hiking in the mountains during an 8.8 earthquake doesn't sound safe at all, does it? At the time of this writing, I still haven't heard anything about her safety, one way or the other.</p> <p><a href="">sirsean</a>: <strong>Hawaii will be impacted by these waves for several hours or days. I should have bought more candy bars before evacuating.</strong></p> <p>Given that the waves were traveling at 500 MPH, the fact that they estimated the first wave to arrive at Hilo at 11:05 AM and at Kauai at 11:40 AM confused me.</p> <p><a href="">sirsean</a>: <strong>Further delay: the wave isn't getting to Kauai for an extra 40 minutes.</strong></p> <p>But I soon learned that the waves travel quickly when they're in deep water, but as they get shallower they slow down tremendously. Also, the reefs surrounding the islands would slow them down even more. They ended up slowing the waves down more than anyone seems to have anticipated.</p> <p><a href="">sirsean</a>: <strong>Nothing happening yet in Hilo. The tsunami must be traveling Delta.</strong></p> <p>It's funny, I think, that I was feeling annoyed that the tsunami was late. Of course, the reporters on TV were talking about how great the estimates were, and how hard it is to determine the exact direction and velocity of the waves, and even getting it within an hour is an amazing feat. I wonder if they were just blowing smoke up the collective asses of the scientists they'd been talking to -- if they'd ripped the accuracy of the reports they may have lost access to the scientists for the next emergency.</p> <p>Still, it's better to estimate early than late.</p> <p><a href="">sirsean</a>: <strong>The Red Cross is here, leisurely setting up.</strong></p> <p>They sent one pickup truck, with an empty back, and 2-3 people. No food, but they did have some extremely sugary juice in big jugs (the jugs were supplied by the school). I took one sip of the juice and practically had to spit it out. You can't drink that crap if you're trying to stay hydrated. Plus, I was pretty surprised at their lack of urgency. It was pretty close to the estimated time of arrival.</p> <p><a href="">sirsean</a>: <strong>Just heard the water is receding. Danger could be approaching soon, probably.</strong></p> <p>I gather that the water rushes out right before the wave gets there. That makes sense, and the TV was making a pretty big deal about it.</p> <p><a href="">cinatyte</a>: <strong>Everyone follow @sirsean. He is in Hawaii and is tweeting constantly about his experiences.</strong></p> <p>I don't know who Mac Wilson (@cinatyte) is, but he sparked a wave of new followers for me. That message was retweeted several times, and the deluge of new followers was begun. At this point I was feeling like a vital part of the newsmedia, disseminating information in real time across the globe. It's an empowering feeling, even if it is, for the most part, merely an illusion. After all, even with the 40 or so new followers I got that day, I still topped out at just 105. That's a pretty small reach for a "vital" news source. Or any news source.</p> <p><a href="">sirsean</a>: <strong>The attitude here at Kapaa Middle School remains a mixture of calm and apprehensive. I'm pretty impressed by the people here, not panicking.</strong></p> <p>Normally I would have said I was impressed by the Hawaiian people, but almost everyone at our shelter was a tourist. I have no idea why they were all so calm.</p> <p><a href="">sirsean</a>: <strong>1st hit reported at just a few centimeters. They've been warning all day that the 1st wave may not be the biggest. Still, somewhat promising</strong></p> <p>All day, they'd been talking about a much bigger wave. My brother had told me via SMS that CNN had estimated 8 feet. The TV had been saying 5-6 feet when it reached Hawaii. A few centimeters, compared to that, is pretty tiny. Also, it amused me that suddenly they switched their units of measurement. You know, in case nobody was already confused.</p> <p><a href="">sirsean</a>: <strong>Water has receded 1-1.5 feet now. That's less encouraging.</strong></p> <p>Reports were coming in from all over the place. Some people were listening to the radio on their headphones and telling me what they were saying. Others were on the phone with people on other islands or back on the mainland who were watching television. I was watching TV myself, and also looking out to sea. Many of the reports seemed to conflict, but I'm sure they were all true at least on some level. After all, the reefs and level of shoaling at different spots on the islands would have vastly different effects on the waves.</p> <p><a href="">sirsean</a>: <strong>Met some German guys, robbed last night at 2am. They have no stuff, no shoes, no money. Talk about a bad day! Those guys need your thoughts.</strong></p> <p>Those guys, who seemed somewhere between 18 and 30 years old, had one of the worst stories of the day. They were visiting from Germany for a month, and had been in Hawaii for a week already. They'd been sleeping on the beach when someone robbed them and took everything they had, including all their cash and their shoes. They tried to report it to the police, but were told that a tsunami was coming and they had to be moved away from the beach and that the police didn't have the time or resources to investigate the crime.</p> <p>They had cuts on their feet from walking over rocks and pavement and such, and they asked the Red Cross for help. The Red Cross had no sterilization supplies, no bandages, and no footwear of any sort. These poor guys were out of luck. I felt terrible for them; even if the tsunami wiped out my hotel and destroyed the stuff we'd left there, we were going to be fine. Meanwhile, even if the tsunami turned out to be nothing, these guys were still screwed.</p> <p><a href="">sirsean</a>: <strong>A pretty heavy, steady wind is picking up from the southeast. Don't know if it means anything, but it might.</strong></p> <p>I still don't know if that means anything, but that wind didn't die down until after the first series of waves had passed, and shortly after it was gone they ended up calling the all clear. So if we think correlation is at all significant, maybe that wind does mean something.</p> <p><a href="">BreakingNews</a>: <strong>5.6-foot #tsunami wave recorded at Hilo Bay of Hawaii, Pacific Tsunami Warning Center tells NBC News</strong></p> <p>It's funny, I got that report at the same time as another one that said the first 2-3 waves had already passed all the islands, and the TV was still showing nothing happening at Hilo.</p> <p><a href="">sirsean</a>: <strong>Just heard a rumor that the first 2 waves have already passed all the islands. There could be many more, but we couldn't really see anything</strong></p> <p><a href="">sirsean</a>: <strong>Water is receding on the east coast of Kauai now.</strong></p> <p>I couldn't see this, despite facing directly at the east coast of Kauai. I suppose it's tough to tell if water has receded a few feet when you're a mile from the coast and don't have a good angle. I'd been hoping to be able to see something.</p> <p><a href="">sirsean</a>: <strong>From TV: "Bottom line, the surfing is very, very good right now."</strong></p> <p>They really were talking about surfing a lot ... but I don't know if <em>that</em> is what I'd call the "bottom line" <em>while a tsunami is currently hitting the island</em>.</p> <p><a href="">sirsean</a>: <strong>A group of people watching the news at the Red Cross HQ here. It's less than 10%; others are scattered around</strong></p> <p><img src="" alt="People watching the news rather than the water" /></p> <p>I called it the "Red Cross HQ," but really that was through the door (to the right of the Dasani machine, behind the guy in the grey t-shirt. They'd set up in there, perhaps to hide from the people who mistakenly thought the Red Cross would have food or other supplies of any kind.</p> <p><a href="">sirsean</a>: <strong>The reef off the NaPali coast of Kauai is completely exposed by all that receding water. It's apparently always covered by water. Wow.</strong></p> <p>They showed a picture of it on the news. It really was remarkable. It made it seem, at the time, like the feeling that the tsunami had already passed was mistaken.</p> <p>But then the emotional roller coaster of the day took another turn.</p> <p><a href="">sirsean</a>: <strong>The police have returned to our shelter, with smiles on their faces. That has to be good news.</strong></p> <p><img src="" alt="Cops returning to the shelter" /></p> <p>I didn't see what the cops did once they got back, but basically it seemed like they walked around for a little bit and then receded back somewhere out of sight.</p> <p><a href="">sirsean</a>: <strong>It seems like some people have already left. There used to be more cars parked here.</strong></p> <p><img src="" alt="Fewer people than before" /></p> <p>I did see some cars driving out, back onto the road, but I didn't personally see very many of them leaving. Plus, I have no idea where they went, because the Red Cross was insisting that all the roads were closed and that we can't leave. Evidently, not everyone cared what the Red Cross had to say.</p> <p><a href="">sirsean</a>: <strong>I can see some of the same discoloration off the Kauai coast that was off Hilo during its surges. Nothing major is happening.</strong></p> <p>Finally, I was able to see some evidence that the water was receding off the east coast of Kauai. The blue of the ocean shifted from a deep blue to a much lighter color, slightly greenish, perhaps aquamarine. There were huge stretches of whitecaps that seemed to just be sitting there, at the line between the blue of the deeper ocean and the green of the shallower water.</p> <p><a href="">sirsean</a>: <strong>People seem a little more relaxed and a little more bored. They're still watching the water, though.</strong></p> <p><img src="" alt="Relaxed and bored" /></p> <p>By this point, I think it'd become clear to everyone that nothing was really going to happen. We were safe, our beds were safe, and the realization was starting to sink in that we'd all sat here calmly, in the sun, for seven hours, without eating any food. (Except for one family that set up right next to the Red Cross HQ and made themselves sandwiches, refusing to share any with anyone else. So they didn't look quite as hungry.)</p> <p><a href="">sirsean</a>: <strong>The tsunami warning has been cancelled. All roads on Kauai are still closed. They're telling us not to leave yet.</strong></p> <p>TV was reporting that there was no reported damage to any property on any island, nor any deaths or injuries. That wasn't surprising in the slightest given the way things had gone, but it was still welcome news. The other welcome news was that the scientists agreed with the feeling of the masses, that the tsunami had passed and that we were not in danger.</p> <p><a href="">sirsean</a> <strong>All clear! We can leave the shelter now.</strong></p> <p>Of course, they told us we <em>can</em> leave the shelter, but we shouldn't because the traffic would be terrible as everyone tries to rush back down the mountain at once. But we didn't have a car, so we weren't worried. We were just going to take a leisurely stroll down towards the water, perhaps going slowly enough that some shops or restaurants might open by the time we passed them, allowing us to eat. (We ended up finding just one open restaurant, a little Chinese place. We did not get the impression that they had closed at all that day, or were aware that there'd been a tsunami scare. In fact, they seemed confused as to why they hadn't gotten any customers all day.)</p> <p><a href="">sirsean</a>: <strong>The principal of Kapaa Middle School did a fantastic job today. He deserves lots of kudos.</strong></p> <p><img src="" alt="The principal of Kapaa Middle School" /></p> <p>After the fact, and with more time to think about it, I still think he did a great job. The police had pointed out that it was a Saturday and he left his family at home to come and help us. The principal pointed out that normally he doesn't dress this way. Blah, blah, blah. The important thing was that he kept everything under control, helped keep everyone calm, supplied televisons and radios, opened more and more of the bathrooms on campus, and generally did a fantastic job. We would have been fine if the Red Cross hadn't shown up, but it would have been a much rougher day if principal Aiwohi hadn't been there. When push was just about to come to shove, he showed that the amazing campus was just the second best thing the students of Kapaa Middle School have going for them.</p> <p>And that, really, is the end of the story. We went back to the hotel, swam in the pool, sat by the beach, watched some pretty impressive waves and the surfers who enjoyed them, and fell asleep early after a long day. We left our clothes nearby, ready to be leapt into at a moment's notice in case there was another tsunami scare in the middle of the night. (Of course there wasn't.)</p> <p>It'd be a better survival story if there'd been at least one person who <em>didn't</em> survive, and a better story in general if there'd been any action. But, ultimately, the story of the day in my mind was Twitter proving itself as a viable and valuable global communication tool. My parents would have been worried sick if I'd been in Hawaii unable to make a phone call while a tsunami raced toward the island. Instead, they were watching my Twitter feed closely, still worried but at least fully aware that I was fine. People interested in following the news don't have to rely fully on the television news any more; they can supplement that with the information and pictures supplied by any number of regular people who happen to be on the ground where the action is happening.</p> <p>In fact, the television anchor said it best:</p> <blockquote><p>"We're going to switch over to show the Olympics, because I know a lot of viewers out there want to see what's going on in Vancouver. But don't worry, there are plenty of websites and Twitterers out there to give you the information you need if you still want to follow the tsunami."</p></blockquote> <p>Or anything else, for that matter.</p> <p><em>Note: Some of the images I posted appear to have cut out midway through the upload. I have decided to leave them like that, to show precisely what I was able to communicate in the heat of the moment, and also perhaps to encourage AT&amp;T to realize that uploading data is at least as important as downloading data in the new era of crowdsourced communication.</em></p> Emitter Progress: Conversations 2010-01-30T00:00:00-08:00 <p>I've been making fairly solid progress on Emitter lately. Today's additions were threefold:</p> <ul> <li>Mentions</li> <li>Replies</li> <li>Conversations</li> </ul> <p><strong>Mentions</strong> work exactly the same as they do in Twitter, and aren't worth talking about here.</p> <p><strong>Replies</strong> are similar to Twitter, but for now there are still some limitations. For one thing, I haven't worked out how to limit the timelines they go into based on the intersection of followers (though that will almost certainly come soon). Another problem: You can currently reply from your home timeline or a list of a user's emissions, but not from an individual emission. But those are simple enough to do later on.</p> <p>But the real interesting thing here is <strong>Conversations</strong>. A conversation can start from any original emission; ie, from any emission that isn't in reply to any other emission. When you emit, there's no conversation yet. But once someone replies to you, a conversation is created and your emission becomes the "original" for the conversation.</p> <p>Anyone who then replies to your original emission <em>or</em> the reply is simply adding to the conversation. This second reply points directly to the emission that it was replying to (whether it was the original or the first reply), as well as to the conversation itself. This data structure allows us to easily get <em>all</em> the emissions in a conversation, or to build out the tree of replies in case we want a threaded display.</p> <p>For now, I haven't figured out a good way to build a UI for this; I have to think about how I want to show a conversation to the user. But I think showing a full conversation could be a valuable way for people to consume emissions; it's something that you kind of have to maintain in your head when you're using Twitter.</p> <p>What do you think about conversations?</p> Open Twitter: What would you name the baby of Email and Twitter? 2010-01-09T00:00:00-08:00 <p>Email is the distinguished, reliable, old man of online communication; Twitter is the hot young coed who draws everyone's eye as soon as she walks into the room. But what if the two of them met and, say, hit it off?</p> <p>People like the immediacy and openness of Twitter, and the asynchronous follower model of online relationships is brilliant. However, all the data is stuck on Twitter's servers, and if you don't like the way Twitter is running their systems you're stuck. Imagine if email had worked the same way: there's only one service, called "" or something, and everyone who wants to send email has to sign up for that one service and can only send email to other people on that same service.</p> <p>It wouldn't be very useful, would it? Opening up email across the internet, and across disparate email servers that only need to know two things about a remote server before allowing you to communicate with someone using it: a) where is it? and b) does it support the same email protocol I do?</p> <p>It seems to me that if the Twitter usage model were opened up such that anyone could run a "timeline server," and you could choose which one you sign up for, and you could follow people on different timeline servers ... well, maybe once you can do that, the usefulness of Twitter will explode in the same way that it did for email.</p> <p>And there are plenty of people pushing for Twitter to adopt new features -- meanwhile, Twitter is going slowly and trying to maintain control until they can figure out how to successfully monetize their currently-strong position with all these users desperate to use their service and stuck there because there are no alternatives. But if there were an open alternative, and different timeline operators could compete for users on reliability, or speed, or cool new features, it stands to reason that things would get better in a hurry. (Do you think the monolithic "" would have an interface as cool as Gmail's? Because, without competition for users, I don't think it's likely.)</p> <p>So I've been playing with such a system for the past week or so. I've got it so you can follow people across timeline servers, and when they tweet to their timeline server it's forwarded to <em>your</em> timeline server so you can see it in your timeline. Some of the API needs to be fleshed out (especially on finer-grained control over how you get tweets back), but it's getting there. Soon, I'll put it up on GitHub.</p> <p><strong>But before I do that, I need to crowdsource something: the name.</strong></p> <p>I can't think of a good name for this. I was thinking "OpenTwitter" at first, but I don't want to include the word "Twitter" in the name.</p> <p>So here's my questions for you:</p> <ol> <li>Would you prefer to tweet on an open communication platform rather than a single company's product?</li> <li><strong>What name would you give to the baby of Email and Twitter?</strong></li> </ol> <p>Thanks for the help, Internet.</p> Eclipse-free Java & Flex: MXML/Actionscript in MacVim 2009-12-01T00:00:00-08:00 <p>At work, I've been doing Flex &amp; Java work pretty much exclusively for a good long while now, and the standard toolchain for that centers around Eclipse; that's what all of us use at work. But Eclipse is slow as molasses and uses a <em>ton</em> of memory ... and tends to break down if you're running 15+ Tomcat projects and compiling/building several different Flex projects. Last week, Eclipse finally gave up; it crashed, and then it wouldn't start up again. I could have continued fighting with it in a misguided attempt to get back to "normal" -- ie, get back to a slow and painful workflow with repeated interruptions while I wait for Eclipse to finish doing whatever it's doing.</p> <p>Instead, I decided to go for an Eclipse-free workflow.</p> <p>I'm using Maven to compile my code, run my tests, and build my JAR, WAR and SWF files. MacVim is my editor (though regular old vim would work just fine). I keep a Finder window open to my workspace so I can easily see directory structures and open files. And I've written some scripts that take the Maven-built WAR files and publish them to the Tomcat webapps directory so I can run them.</p> <p>I posted a while ago over at my old blog about <a href="">how to get syntax highlighting working in vim for MXML and Actionscript</a>; that solution still works well, but it's important to note that <a href="">vim's syntax files go into a different directory* when you're using MacVim</a>:</p> <pre><code>/Applications/ </code></pre> <p><em>* You will have to right-click and select "Show Package Contents" to get there.</em></p> <p>Here are the links to the files:</p> <ul> <li><a href="">actionscript.vim</a></li> <li><a href="">mxml.vim</a></li> </ul> <p>After putting the two files in the proper location, the following lines go into the ~/.vimrc file:</p> <pre><code>au BufNewFile,BufRead *.mxml set filetype=mxml au BufNewFile,BufRead *.as set filetype=actionscript syntax on autocmd FileType mxml set smartindent autocmd FileType as set smartindent </code></pre> <p>Frankly, I think this gives vim <em>at least</em> as much syntax-highlighting capability as Eclipse/FlexBuilder <em>ever</em> had, and I don't actually find myself missing the code completion all that much. (If anyone knows of any good code completion solutions for MXML/AS beyond the simple ctrl-N ... let me know.)</p> <p>So far, my Eclipse-free existence has been going brilliantly. I'll try to post a few more times, fleshing out a little bit more about the environment.</p> Eclipse-free Java & Flex: Deploying and Running in Tomcat 2009-12-01T00:00:00-08:00 <p>Earlier, I (briefly) described <a href="">my long-running battle with Eclipse, which Eclipse finally won by dropping the proverbial a-bomb and refusing to start</a>. If you want to read that first, be my guest. Here, I describe the next step in setting up my current environment.</p> <p>Part of the deal with escaping from Eclipse is that I no longer get the benefit of its automatic-publishing to a Tomcat container; from now on, I'm going to have to do that myself. Here's how I'm doing it.</p> <p>First, you'll need an installation of Tomcat. I already had this, and I'd put it in my user-specific Applications directory. It's located at:</p> <pre><code>~/Applications/apache-tomcat-6.0.20/ </code></pre> <p>Yours may differ depending on the version ... but I <em>do</em> recommend putting it in your home directory rather than setting it up in a global location.</p> <p>Since we're using HTTPS for our applications, the first thing you need to do is go into conf/server.xml and uncomment the Connector for port 8443:</p> <pre><code>&lt;Connector port="8443" protocol="HTTP/1.1" SSLEnabled="true" maxThreads="150" scheme="https" secure="true" clientAuth="false" sslProtocol="TLS" /&gt; </code></pre> <p>Mine was on line 81 ... yours might be in a different place, but it should be around there.</p> <p>The next thing you need to do is increase the memory allotted to Tomcat to run; this <em>may</em> not be totally necessary, but if you're running a handful or more services you'll probably get some Out Of Memory / PermGen space errors. I know I did.</p> <p>Go edit bin/, and put the following line around line 200 (right before it sets the JAVA_OPTS variable):</p> <pre><code>JAVA_OPTS="$JAVA_OPTS -server -ms512M -mx1024M -XX:NewSize=256M -XX:MaxNewSize=256M -XX:PermSize=256M -XX:MaxPermSize=256M " </code></pre> <p><em>* Newlines added for pretty-printing purposes.</em></p> <p>What this does is take any existing value of JAVA_OPTS and tacks some more options onto it. I set it to a minimum of 512M and a maximum of 1024M, and then set the PermGen space to 256M (the -XX:NewSize, -XX:MaxNewSize, -XX:PermSize, and -XX:MaxPermSize options are responsible for this). You might be able to get away with less than 256M of PermGen space, but I couldn't, and this works. I initially had my minimum memory set to 256M, but that failed to start up because it claimed it couldn't fit its PermGen space into the heap; that's why I bumped the minimum up so high.</p> <p>The remaining options are there for enabling JMX on the server. I have some JMX-enabled services working on there, so that's necessary for my purposes. If you need it too, it's worth noting that this is solely for development; if you're putting this into production <em>you absolutely should <strong>not</strong> turn off the authentication and SSL options</em>.</p> <p>The final step is getting your WAR files into the Tomcat webapps directory. I'm using Maven to compile and build, so I've got my WAR files available in each project's target directory. So, to deploy the WARs, I created some scripts in my workspace. The first one deploys the base services, all together:</p> <pre><code>echo "base-service-one" rm -rf ~/Applications/apache-tomcat-6.0.20/webapps/base-service-one* cp base-service-one/web-app/target/base-service-one-webapp-*.war ~/Applications/apache-tomcat-6.0.20/webapps/base-service-one.war echo "base-service-two" rm -rf ~/Applications/apache-tomcat-6.0.20/webapps/base-service-two* cp base-service-two/target/base-service-two-*.war ~/Applications/apache-tomcat-6.0.20/webapps/base-service-two.war </code></pre> <p><em>* Newlines added for pretty-printing purposes.</em></p> <p>And so on. (Note that your target directories may be in different locations per project, as mine are, depending on what type of project you're dealing with. Be sure to check out each directory structure after Maven's done its thing.) The reason I'm doing "base-service-two-*.war" is because Maven will actually create a file like "base-service-two-0.0.1-SNAPSHOT.war" or something like that, and I don't really care what version it currently is.</p> <p>Then for each individual project, I have a <em>separate</em> deploy script:</p> <pre><code>echo "project-one" rm -rf ~/Applications/apache-tomcat-6.0.20/webapps/project-one* cp project-one/web-app/target/project-one-webapp-*.war ~/Applications/apache-tomcat-6.0.20/webapps/project-one.war </code></pre> <p><em>* Newlines added for pretty-printing purposes.</em></p> <p>The reason for deleting from webapps <em>before</em> copying the WAR over is so that Tomcat knows to re-deploy. I had some trouble with Tomcat detecting new WAR files, so I'm just deleting the old WAR <em>and</em> exploded directory to be safe.</p> <p>Run your deploy scripts after every time you successfully run "mvn install" ... and you're ready to start Tomcat:</p> <pre><code>~/Applications/apache-tomcat-6.0.20/bin/ </code></pre> <p>And, bam. Your services are starting. And you didn't need to touch Eclipse.</p> Python/MySQLdb on Snow Leopard 2009-09-26T00:00:00-07:00 <p>This one is mostly for me, so I remember how I did this. I like developing webapps using <a href=""></a>, which obviously requires Python to be able to talk to MySQL if you want to, you know, store anything.</p> <p>On my old laptop I ran into some difficulty getting that to happen -- for some reason OS X doesn't come with the necessary libraries for Python to talk to MySQL, which seems like an oversight to me -- so I just spun up an Ubuntu VM on which to do my development. (sudo aptitude install mysqldb-python (or python-mysql, or something, whatever it actually is, do an aptitude search first) is pretty damn easy.)</p> <p>But my new MBP came with Snow Leopard, and I wanted to get my development environment working without the VM. So ... first I installed the x86_64 version of MySQL (which says it's for Leopard, but it works fine in Snow Leopard). You also need to have XCode installed to compile MySQLdb.</p> <p>I followed the directions <a href="">here</a> and <a href="">here</a>, because neither one of them ended up working by themselves. My ~/.profile ended up looking like this:</p> <pre><code>export PATH=$PATH:/usr/local/git/bin:/usr/local/mysql/bin export CC="gcc-4.0" export CXX="g++-4.0" </code></pre> <p>Note from the first link that step 4 appears to be unnecessary in the version of MySQLdb that I downloaded (1.2.3c1), which is newer than the one he used (1.2.2). The C code changes were already done.</p> <p>I don't know if the symlink and changes were important, but I did them anyway, because what's the big deal?</p> <p>Anyway, now I don't have to bookmark those links.</p> The Perch Roulette Simulator 2009-08-30T00:00:00-07:00 <p>So I was drinking off another demoralizing Twins loss on Saturday night, and bemusedly clicking around my Google Reader feeds which have somehow been growing and growing unchecked for the last few weeks. Having not read The Daily WTF for a while, I figured I'd try to clear some of them out. I came across one called <a href="">Knocking Me Off The Perch</a>, which involved a programmer and a lawyer who walked into a casino,* and ended up inventing an algorithmic roulette strategy wherein they turned $10 into $400 simply by following a few easy steps.</p> <p><em>* Have you heard this one before?</em></p> <p>Well, here are the steps, as they described them:</p> <blockquote><p>Then we realized something. We had just discovered The Perch, a roulette strategy that could not fail. The rules were defined as follows.</p> <ol> <li>Perch someplace where you can see a number of roulette tables at once.</li> <li>As soon as a table shows four consecutive blacks or reds, swoop in and place a bet.</li> <li>If the bet is successful then return to the Perch. Otherwise, place another bet worth 150% of the previous bet (instead of $10, place $15).</li> <li>If the second bet fails, then shrug your shoulders and return to the Perch.</li> </ol> </blockquote> <p>The programmer ended up concluding, the next morning, that there is no strategy for winning at roulette. The lawyer maintained that there was, and that they'd discovered it.</p> <p>None of this was particularly interesting, until the "Bring Your Own Code" section where they challenge you to write your own simulator to see just how rare it is to do what they did.</p> <p><a href="">So, I did.</a></p> <p>According to my simulator, if you use this strategy you'll turn $10 into $400 a little over 50% of the time. Which doesn't sound bad, except that the games where you actually win and get your $400, you're standing there for over 330 spins of the roulette tables. Oh, and in the games you end up losing all your money? Well, those ones only last about 8 spins.</p> <p>My first question is: If you have a 50% chance of increasing your money 40x over ... do you do it? (Even if it takes several hours?)</p> <p>My second question is: Is there anything wrong with my simulator? I can't imagine that casinos would allow such a <em>huge</em> arbitrage situation to exist on the floor.</p> googlevoicenotify - A cool idea, but the execution isn't there yet 2009-08-11T00:00:00-07:00 <p>So a friend of mine told me about this cool thing you could do with Google Voice and Prowl: poll Google Voice for new SMS messages and send them to your iPhone with push notifications. It'd be a cool way to cut out all incoming and outgoing text messages -- and thus save some money on your plan.</p> <p>He told me about <a href="">a Python program that'd act as a daemon</a> that polls Google Voice, parses out the text messages, and sends them along to Prowl. Sounds like it'd work great, right?</p> <p>Well, there were some problems.</p> <p>The first problem is that it relies very specifically on a certain HTML schema* from Google Voice -- one that has a div element with a class of "gc-message gc-message-sms" <em>and</em> an id attribute that uniquely identifies a "thread" of conversation. This wouldn't be such a problem ... if the Google Voice "API" actually supplied its information in that format. There is no such thread-wrapping div, and there does not appear to be any unique identifiers for any threads. At first I only had one thread, so I thought that might have been the problem; so I texted my GV account from another phone, creating two threads. There are still no wrapping divs, and still no thread identifiers (unique or not). So ... that's a huge problem.</p> <p><em>* The last commit to this project was on July 31, and I can only assume that the author had it working for himself at the time. Which means that in the last 10-12 days, Google has changed the schema for this page fairly significantly. I wouldn't be at all surprised if it continued to change, perhaps at a fairly rapid pace ... which says to me that this isn't going to be a good idea any time soon.</em></p> <p>Anyway, that's a fundamental problem with dealing with Google, and the fact that they don't really give a shit about you. But there's another problem, and that's with the design of the application.</p> <p>Since Google's "API" doesn't specify which SMS messages are "new," and there's no way to mark them as "read," you get <em>all</em> of them at once every time you poll. Given that you don't want to get a push notification for <em>every</em> text message you've <em>ever</em> received every 30-60 seconds or so, the program has to determine which ones are new. And it doesn't do that very elegantly.</p> <p>It creates a dict of threads, keyed by the thread id, each of which are an array of text messages. It then pickles this dict and saves it to a file on the filesystem. That has to be pickled and unpickled, obviously, which takes time <em>and gets slower with each text message you ever receive.</em> But the dict of threads itself can be accessed in constant time ... that's not true of the array of messages in each thread. In order to query that, you have to use the "message identifier" to see if the message exists in the thread, and that's an array-in call -- linear time! Woo!</p> <p>Can it get dumber than that? Yes. Yes it can. I haven't told you what the "message identifier" is. Well, Google doesn't supply an actual identifier ... so the author of this program decided that the unique identifier of a text message would be ...</p> <p>Drum roll please.</p> <pre><code>identifier = from_name + ' ' + message_txt </code></pre> <p>Yes. That's right. I hope nobody you know ever sends you a text message with the same content. Ever. You know, like "Where are you?" or "What?" or something along those lines. I get (and send) those all the time ... and with this program you would only be notified the first time someone <em>ever</em> sends you one. (Unless there's something about the way Google defines "threads" that <em>guarantees</em> that there can never be two messages in a thread that have the same content. I doubt it.) So, yes. Unacceptable.</p> <p>What amuses me even more is that Google <em>does</em> supply something that can come pretty close to guaranteeing that you can get those repeat messages -- there's a time field, with precision to the minute. So unless someone's sending you messages with the same content multiple times per minute (certainly possible, but you may not <em>need</em> notifications for every one of them, and also your friends are boring), you'll get them.</p> <p>So it gets slower linearly with the number of text messages you've ever received, and it becomes increasingly likely to drop messages as time goes on. I'm going to go ahead and call bullshit on that.</p> <p>The way you want to improve this is to create an SQL table with (from, text, time) as the only fields, and make it unique across all three fields. For each message you parse, you first check your database to see if the message exists. If it does, ignore it. If not, insert it and push the notification; lather, rinse, repeat until all the messages have been checked. It ignores the thread problem, and works with the <em>current</em> schema from Google. It also doesn't slow down linearly -- and since your database is local, it'll take a while before those queries start to slow you down (and it'll certainly scale better than reading/writing an ever-growing file and doing in-array operations hundreds of times in a loop.</p> <p>If I can make some time for this, I'll go ahead and fork the project and make these changes. But if someone else does it before I can, do let me know.</p> Switching from the visual editor to Markdown 2009-08-08T00:00:00-07:00 <p>So I've decided to start using <a href="">Markdown syntax</a> to post on here, as a nice little test. The visual editor has been getting more and more annoying, and I want to see how well this works.</p> <blockquote><p>The idea for Markdown is to make it easy to read, write, and edit prose. HTML is a <em>publishing</em> format; Markdown is a <em>writing</em> format. Thus, Markdown’s formatting syntax only addresses issues that can be conveyed in plain text.</p></blockquote> <p>Sounds good to me.</p> <p>And if it works out here, I might start using it on the other blogs. Which would be nice ... I don't really want to have a radically different editor for each different blog. If I can post enough on here to be able to tell how it's working, I'll probably mention how the transition goes.</p> <p>That is all.</p> ZFS sounds like fun 2009-05-16T00:00:00-07:00 <p>I just read an article on some guy's blog about <a href="" mce_href="">using Solaris and ZFS as a home file server</a>.</p> <p>Frankly, it looks awesome. I've been wishing for some ZFS goodness for a while, and could probably come up with some ways to use up a few terabytes of space.</p> <p>My only question is ... how do I justify spending $1300 on something that I have to invent a use for?<br /></p> <p style="font-size: 10px;"> <a href="">Posted via web</a> from <a href="">sirsean.posterous</a> </p> Yes this is happening 2009-05-03T00:00:00-07:00 <p>Good weather on a Sunday afternoon, Twins game is on ... Sounds to me like a good excuse to fire up the grill.<p><a href=''><img src="" width="500" height="375"/></a> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Grilling again 2009-04-30T00:00:00-07:00 <p>There's some tasty-looking chicken on the grill. <br />&nbsp;<br />What is it about Thursdays that bring out the best in my grill?<p><a href=''><img src="" width="500" height="375"/></a> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Bam. More blooming! 2009-04-28T00:00:00-07:00 <p>Today the amaryllis went from 2 flowers to 4, and they look like they're getting both bigger and redder. <br />&nbsp;<br />And just for the record, there's another stalk that's really shooting up right now. Certainly could be blooming in the next couple of days.<p><a href=''><img src="" width="500" height="375"/></a> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> People will pay for online content if you make it easy enough 2009-04-26T00:00:00-07:00 <p>Posnanski's future of newspapers blog has a post up by Seth Mnookin about what the newspaper companies (or anyone else, really) can do to <a href="" mce_href="">convince people to pay for content</a>. His basic point is that it's not that people don't <i>want</i> to pay for content; instead, they just don't <i>want</i> to enough to jump through all the extra hoops required to pay. It's so much easier to get the content for free that that's the way people do it.</p> <p>He points to Apple's iTune Store and iPhone App Store, and to Amazon's Kindle as evidence that people are more than willing to pay for content if you make it easy, simple, and remove the extra step of paying every single time.</p> <blockquote><p>the news industry's "original sin" wasn't so much giving away content for free, it was making it so damn hard to pay for content. I contrasted that with two popular topics here: the Kindle and the iPhone. Amazon and Apple have both perfected the type of instant-gratification, on-the-spot payment plans that basically erase the lag time between wanting something and owning it--buying a book or an app are, in today's parlance, incredibly low-friction transactions.</p></blockquote> <p>His solution is that the newspaper companies should form some sort of coalition, a gateway through which people can easily purchase content from any of the participating newspapers/magazines/publications and get it immediately, without having to enter their credit card information for each purchase.</p> <p>I think the idea makes a lot of sense, and could certainly work. But what both Apple and Amazon also have going for them is a piece of proprietary hardware that makes the whole operation extremely smooth and seamless. The publishers <i>definitely</i> need to make this work online on any computer, and they should have an iPhone app and find a way to get onto the Kindle.<br /></p> <p>But I'd say they should also build a piece of hardware that they can sell at a small profit which is designed specifically for this service, and simply works better, faster, and more seamlessly than the web/iPhone/Kindle versions.* That could really work.</p> <p><i>* Of course, they absolutely must not cripple the web/iPhone/Kindle versions in order to do this. You don't want to artificially make the majority of your customers second class citizens if you can help it. The goal here would be to make the specialized reader hardware <b>so good</b> that people are willing to pay $200 to get it rather than paying $5 for the iPhone app or $0 for the web version.</i></p> <p>I don't think the newspaper companies will actually pull this off. And if they do, I don't expect they'd get it right. But they definitely need to give it a shot.<br /></p> <p style="font-size: 10px;"> <a href="">Posted via web</a> from <a href="">sirsean.posterous</a> </p> Amaryllis partner 2009-04-26T00:00:00-07:00 <p>I went to bed at 3am, and this hadn't started yet. <br />&nbsp;<br />But when I woke up this morning, the amaryllis plant had a nice little partner.<p><a href=''><img src="" width="500" height="375"/></a> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Amaryllis in the morning 2009-04-25T00:00:00-07:00 <p>I showed you what it looked like last night when it had just started to bloom. <br />&nbsp;<br />Well, it's now had a whole night to think about what it's done, and it look like it's decided to open up. <br />&nbsp;<br />If it gets any bigger or redder, you'll be the second to know. (I, naturally, will be the first.)<p><p><a href=''><img src="" width="375" height="500"/></a></p> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Darlene's Amaryllis 2009-04-24T00:00:00-07:00 <p>After growing over 2 feet in the last 48 hours, it's starting to bloom. <br />&nbsp;<br />I think it's looking pretty impressive.<p><p><a href=''><img src="" width="375" height="500"></a></p> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Still looks tasty 2009-04-23T00:00:00-07:00 <p><a href=''><img src="" width="500" height="375"></a></p> <p> <p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Posterous formatting 2009-04-23T00:00:00-07:00 <p>I've noticed that when you have Posterous autopost to a Wordpress blog, the formatting doesn't really work out that well. At least from Gmail. <br />&nbsp;<br />The iPhone mail client seems to work just fine, though that's probably because you can't do much formatting. In Gmail, if you turn off formatting and just do plain text, it's a little better, but it puts in line breaks where it shouldn't. <br />&nbsp;<br />I find this pretty irritating. It basically means that I'll only be able to use Posterous as a mobile blogging service. Which I guess I don't mind. I couldn't do that before. <br />&nbsp;<br />Still, I wish the Gmail-Posterous-Wordpress workflow worked better.<p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Firing up the grill for the first time 2009-04-23T00:00:00-07:00 <p>It's beautiful, isn't it?<p><p><a href=''><img src="" width="500" height="375"></a></p> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Also chicken 2009-04-23T00:00:00-07:00 <p>We're not just cooking short ribs tonight, here's some delicious looking chicken that'll go well with the rest of our "meal."<p><p><a href=''><img src="" width="500" height="375"></a></p> </p><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Writing 2009-04-22T00:00:00-07:00 <p>I&#39;ve long noticed that when I&#39;m writing an email, my voice comes out better than when I&#39;m writing a blog. It seems to me that this has to do with the idea of an intended audience -- when writing an email, I&#39;m writing <i>to</i> someone, but when writing a blog I&#39;m just writing and the audience is &quot;everyone.&quot; (Though in reality it&#39;s more like &quot;no-one.&quot;)<div> <br /></div><div>I&#39;ve had this thought in the past and it&#39;s never made any difference for me. Whenever I sit down and start writing a blog entry, my &quot;voiceless&quot; voice comes out and it doesn&#39;t sound like me any more. Personally, I find this unacceptable. I simply need to do a better job of writing blogs like I do emails, even if I come off more insulting, less understandable, and more opaque. I think it&#39;ll be for the best.</div> <div><br /></div><div>Most people I read online don&#39;t seem to have this problem, which leads me to wonder what my problem might be. I guess it&#39;s that to me this feels like &quot;public speaking,&quot; as opposed to &quot;talking to somebody you know,&quot; and I see a big difference between those two acts<em>.</div> <div><br /></div><div></em> Although I&#39;m good at neither, just for the record.</div><div><br /></div><div>Maybe what I need to do is not &quot;pretend&quot; that I&#39;m talking to one person, but instead just talk for myself. As if, say, I&#39;m talking to myself. I do that often<em>, and maybe that&#39;d be the best way to bring out my voice in online writing.</div> <div><br /></div><div></em> In my head. I&#39;m not crazy.</div><div><br /></div><div>I don&#39;t think there&#39;s a single solution to this issue (is there ever?), but I want to solve it for myself so my online writing doesn&#39;t seem so foreign to me.</div><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Thoughts on Posterous and Friendfeed 2009-04-22T00:00:00-07:00 <p>As I try to start using Posterous, my thoughts about it keep going back to another service I've been trying to start using: Friendfeed. <br />&nbsp;<br />With Friendfeed, you just tell it where you're publishing stuff online, and it goes out and aggregates it for you on one place. It's pretty cool, and makes it easy for anyone to find all the stuff you're producing. But it's also extremely easy to set it and forget it. Beyond the first 10 minutes or so, it requires absolutely zero engagement. <br />&nbsp;<br />Posterous is close to the exact opposite. It pushes things out to other places online, if you want, but only if you actually produce the content for Posterous first. You actually have to actively use it in order to get any benefit. And they've done a remarkable job of making that required engagement seamless. <br />&nbsp;<br />I like being able to send an email and have it go to whichever blog I wanted it to. And I like being able to do it on my phone. My content ends up on the sites it would have anyway, Posterous gets to keep a copy, and a link shows up aggregated at Friendfeed. Everything works out nicely for me. <br />&nbsp;<br />But which company is built to last? I'm not much of a prognosticator, and my predictions are almost always wrong. Worse than a coinflip, anyway. <br />&nbsp;<br />That said, these are two hugely different models, and I think both are really interesting. They both get access to a lot of online content, and they get it instantly. They don't have to crawl anything like Google does. <br />&nbsp;<br />I wonder if Friendfeed's lack of engagement might hurt it, but you can use Posterous without ever going to their site too. <br />&nbsp;<br />I don't have enough (e-)friends to be able to tell how each of them do from a social networking perspective. I do know that in the brief time I poked around, I found a little more interesting content on Posterous than I did on Friendfeed, but that's probably just random. <br />&nbsp;<br />I haven't heard anything about Friendfeed trying to make money, but Posterous claims to be working on freemium plans, and it's extremely obvious that there are places they could easily charge money. So that'll probably help them as a company in the long run. <br />&nbsp;<br />I think it's nice that Friendfeed pulls in tweets and videos that I put online elsewhere, rather than requiring me to go through their system. That said, it's also nice that my Posterous isn't littered with all my stupid tweets. This is probably a wash. <br />&nbsp;<br />I don't know which will last. They both might be successful, which would be great for me, given that I expect to continue to use both. <br />&nbsp;<br />And they're both more compelling than Twitter. I doubt I could have expressed all this in 140 characters. <br />&nbsp;<br />The bus is almost to its destination, so I'll sign off now.<p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Perens "Says" Fear the Cloud 2009-04-22T00:00:00-07:00 <p>Bruce Perens has an interesting article up about an apparent attack on a smallish town called Morgan Hill&#39;s modern infrastructure. Coordinated attackers cut the communication lines, cutting out phones, cell phones, internet, security alarms, et cetera, all at once. It&#39;s a somewhat frightening tale of just how vulnerable our civilization is to a deliberate and semi-sophisticated attack.<div> <br /></div><div>In the middle, he snuck in a couple of throw-away paragraphs about the dangers of software-as-a-service, and cloud computing, though he couched his concerns as lightly as he could.</div><div><br /></div><div> <blockquote style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; "> This should lead managers of critical services to reconsider their dependence on software-as-a-service rather than local servers. Having your email live at Google means you don&#39;t have to manage it, but you can count on it being unavailable if your facility loses its internet connection. The same is true for any web service. And that&#39;s not acceptable if you work at a hospital or other emergency services provider, and really shouldn&#39;t be accepted at any company that expects to provide services during an infrastructure failure. Email from others in your office should continue to operate.</blockquote> <div> </div><blockquote style="margin-top: 0px; margin-right: 0px; margin-bottom: 0px; margin-left: 0.8ex; border-left-width: 1px; border-left-color: rgb(204, 204, 204); border-left-style: solid; padding-left: 1ex; "> What to do? Local infrastructure is the key. The services that you depend on, all critical web applications and email, should be based at your site. They need to be able to operate without access to databases elsewhere, and to resynchronize with the rest of your operation when the network comes back up. This takes professional IT engineering to implement, and will cost more to manage, but won&#39;t leave you sitting on your hands in an emergency.</blockquote> <div><br /></div><div><a href=""></a> </div><div> </div><p style="margin: 0.0px 0.0px 16.0px 0.0px; font: 16.0px Times"><span style="font-size: small; "><font face="arial, helvetica, sans-serif">I think this is all true, and people should continue to be worried about the cloud, and what happens when the service (or the network) becomes unavailable. You don&#39;t want to be dependent on outside services for things that are important to you<em> or your company.</font></span></p> <p style="margin: 0.0px 0.0px 16.0px 0.0px; font: 16.0px Times"><i><span style="font-size: small;"><font face="arial, helvetica, sans-serif"></em> As a matter of fact, yes, I am aware of the irony of someone posting an article like this by writing it in Gmail, sending it to Posterous, having it stored at Dreamhost, and ending up at Friendfeed. And a Twitter notification will go out announcing the post! I don&#39;t care. I still don&#39;t trust the cloud.</font></span></i></p> <p style="margin: 0.0px 0.0px 16.0px 0.0px; font: 16.0px Times"><span style="font-size: small;"><font face="arial, helvetica, sans-serif">And hopefully nobody who hears about this decides to use the knowledge maliciously. Because I&#39;d be really annoyed if someone cut out my internet for a few days.</font></span></p> </div><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Untitled 2009-04-21T00:00:00-07:00 <p>I've discovered how you can post to different blogs, at will, through Posterous. And frankly, I think it's fantastic. I look forward to a lot more posting of crap from my iPhone, because let's go ahead and be honest: 140 characters isn't enough space. It's much easier to open up the email app and throw down whatever I feel like saying without having to worry about that pesky little number in the corner telling you how many more times you're allowed to hit the buttons. <br />&nbsp;<br />The best thing about computers is the ability to hit buttons, and I don't think that ability should be limited in any way. Especially artificially. <br />&nbsp;<br />I'm not going to lie, I'm really enjoying Posterous so far. It seems to me that a lot more people should be using this.<p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">sirsean.posterous</a> </p></p> Testing sending posterous to a blog 2009-04-21T00:00:00-07:00 <p>This annoying rainout at Fenway has given me the opportunity to try some things out, and one of them is getting to know Posterous. One of the main things I want from it is to be able to post from it to multiple blogs, and being able to choose which one to post to. <div> <br /></div><div>I&#39;ve set up autoposting to two of my blogs thus far, and I&#39;m somewhat curious as to which ones (if any) this will get posted to.</div><div><br /></div><div>After this I&#39;ll be trying to control which blog it goes to. If it can do that, I&#39;ll be pleased.</div><p style="font-size: 10px;"> <a href="">Posted via email</a> from <a href="">Sean's posterous</a> </p></p> When Will People Realize That the Demise of Newspapers is a Good Thing? 2009-04-05T00:00:00-07:00 <div> For some reason people ascribe some sort of magical significance to "newspapers," as if there can't be "news" without "paper," and if information is distributed in some non-tree-killing fashion, the world is sure to collapse post-haste. In a <a href="">strangely anti-Google-and-all-those-nerds-on-the-internet screed</a>, "Henry Porter" drops this little nugget: <blockquote>In 1787 Thomas Jefferson wrote: "Were it left to me to decide whether we should have a government without newspapers or newspapers without a government, I should not hesitate to prefer the latter." A moment's thought must tell us that he is still right: newspapers are the only means of holding local hospitals, schools, councils and the police to account, and on a national level they are absolutely essential for the good functioning of democracy.</blockquote> Perhaps, but two moments of thought would tell you that it's not the mystical process of printing ink onto paper that keeps government functions accountable, either on a local or national level*. It is in fact the content that is created, and the fact that it is distributed to the people to whom it's important and useful. <em>* Don't services like YouTube and all the various blogs out there that shed light on police injustices (along with photo and video evidence) kind of put the lie to this theory? It seems kind of obvious the internet offers a much more effective way of distributing information and protecting citizens' rights than anything the newspapers were ever able to do. Can't "websites" offer local information about hospitals, schools, councils, and police in the same way -- or better -- than "newspapers" could?</em> Three moments of thought might get you to realize that it simply makes no sense to print information, repeated millions of times per day, every day, with low quality ink on low quality paper, and pay people to manually deliver it to every home? Especially when that same content can be beamed into those same homes, at nearly no cost, without cutting down trees or wasting money? Do you think that if Gutenberg had had the option of beaming his content to every house in Europe rather than painstakingly setting his type and distributing the resulting paper, slowly and expensively to just a few choice people, that he wouldn't have jumped at the chance? I contend that even the guy who invented the printing press doesn't think there's anything particularly magical about putting ink on a paper. <blockquote>Google is in the final analysis a parasite that creates nothing, merely offering little aggregation, lists and the ordering of information generated by people who have invested their capital, skill and time.</blockquote> I find this particularly stunning. The obvious implication is that Google never created anything, and that the creation of those aggregators, lists, and ordering of information required no investment of capital, skill, or time. Indeed, only the special process of creating content requires skill ... things like computer programs and printing presses just spring fully formed from the earth to allow leeches to steal things of value. Right? <blockquote>[The newspaper business] now finds itself laden with debt (not Google's fault) and having to give its content free to the search engine in order to survive. Newspapers can of course remove their content but then their own advertising revenues and profiles decline. In effect they are being held captive and tormented by their executioner, who has the gall to insist that the relationship is mutually beneficial. Were newspapers to combine to take on Google they would be almost certainly in breach of competition law.</blockquote> This is just shockingly wrongheaded, in every way. But I'm curious to know what the newspapers could possibly do to "take on Google" to prevent readers from being able to find their content. If they managed to do that, would they "win?" What is it about "creative" types that they can't understand that what's important is the content they create, not the way it's distributed? The internet is a much better information distribution mechanism than newspapers ever were, and will be able to fully replicate everything that newspapers were able to offer, plus more that newspapers could never dream of. It may not be mature yet, and copyright implications and author-payment haven't been figured out yet. But the current state of the internet is very similar to how the newspaper industry was in its infancy, and we managed to figure that out pretty well. <blockquote>We could do worse than follow their example for this brat needs to be stopped in its tracks and taught about the responsibilities it owes to content providers and copyright holders.</blockquote> That just isn't very helpful. What's a little more helpful is something like <a href="">this</a>: <blockquote>We're back to 1836 now, in a sense; everybody who wants to has his own "newspaper", and it's tough to know who is good and who is reliable and who isn't, but the same processes are still running. The blogs will get bigger; the good ones are hiring a second helper and a third and fourth, and we'll spend a century or more sorting things out and re-creating the market. It's hard, but it's not a bad thing. It's a good thing.</blockquote> Anyone who's interested in this should definitely read that article. Bill James knows his stuff. What we need is to stop fighting the demise of "newspapers," and start figuring out how to translate their best features to this new, superior medium.</div> Eclim Disappointment 2009-04-04T00:00:00-07:00 <p>For a while, I've been wishing that I could use a Vim editor inside Eclipse. I'm basically forced to use Eclipse at work, but editing text in Vim is just so much more pleasant. For many months, I've just lived with the Eclipse text editor. Until yesterday, when I was alerted to the existence of Eclim, which promises to allow me to use Vim as an editor within Eclipse, and claims to offer many interoperability features which allow me to get all the code-completion benefits of Eclipse and all the text editing benefits of Vim. Sounds like a perfect solution!</p> <p>Well, I didn't want to potentially break my work environment, so I waited until I got home, and opened up my Windows XP machine (to more closely mimic my work laptop, which is also unfortunately a Windows machine). I installed GVim, which is apparently the style of the time in Windows land. I then opened up the Eclim installer and followed the rather simple directions.</p> <p>It only took a few minutes, and I had the eclimd server running from within Eclipse. I grabbed a file and opened it up with Vim, and it opened right inside Eclipse ... exactly what I wanted!</p> <p>Right?</p> <p>Well ... no.</p> <p>Unfortunately, at least on Windows, Vim uses the hideous system console font by default. You can change it ... but it doesn't save the new font between files, or even between instantiations of Vim. You have to change the font within Vim each time you open a file, even if you're opening a file you've previously opened and set the font. <em>Maybe</em> there's a way to permanently set the font ... but there doesn't appear to be an option for it. Either way, why would it be this difficult/impossible to change the default font, and why would said default font be the worst one this side of Comic Sans?</p> <p>Secondly, the Vim editor requires focus before you can actually use it ... but Eclipse retains focus after opening a file. So after opening a file, you have to click in it in order to start working. Annoying. But that's just step one of the total lack of integration between Vim and Eclipse. The code completion does not work, and it doesn't automatically include imports (which was one of my main desired features for this). You can't switch editor tabs via the keyboard, and none of the Eclipse keybindings work from within the editor. You <em>can</em> muddy up your .vimrc to get this "working," but I don't necessarily want to do that to my.vimrc ... and .vimrc's nmap command doesn't appear to support multiple-control-key bindings (ie, Ctrl-Shift-O does not work, you have to come up with some other keystroke for that functionality ... not that that particular one would even work).</p> <p>If you hit the big X on the tab to close the window, it'll throw up a null pointer exception for you before actually closing the editor tab. Which would be annoying, except that re-opening that file is even worse. It throws another null pointer exception, with <em>another</em> modal dialog behind it explaining that there's a Vim swap file for that file, giving you the option of editing anyway, restoring or deleting the swap file, etc. So you have to remember that not only can you not use Ctrl-W to close a tab (at all), you can't even close it via the GUI. You <em>must</em> use :q to close an editor tab.</p> <p>Oh, and the "Outline" view doesn't work if you use the Vim editor; apparently that feature of Eclipse interacts with the Java editor, as does the constantly-compile-my-source-code feature -- it doesn't compile until you save the file.</p> <p><em>Maybe</em> this eclim plugin works better in OS X and Linux, where both Vim and interprocess communication are a little more natural than they are in Windows. But frankly, I don't feel like finding out. I have to use it at work anyway, where for some reason I don't have a Mac. And this experience has been annoying enough that I'm not even interested any more.</p> <p>Eclim does not work, and is a huge disappointment.</p> Signed up for the Palm Mojo SDK 2009-04-02T00:00:00-07:00 <p>So I signed up for the Palm WebOS Mojo SDK preview today. They asked what application I'm planning to write once I get it, so I told them about a mobile client for Things I Did. I think it'd be cool, and I was planning to make such a client once I got the Pre; my main fear is that they'll be as apathetic toward that application as everyone else is. Either way, I'm still pretty excited to get the Pre.</p> <p>Also, I got around to watching the Pre walkthrough video they released this week. It still looks a little buggy, and it also looks just as slow to open applications as the iPhone is. Hopefully it's smoother to use than it looks in the videos. I think I'm going to have to use one in a store before I actually buy one. That's something I've never done for a phone before, including the iPhone.</p> <p>Will I get access to the Mojo SDK? Stay tuned.</p> Flex 3.1.0 Bug, FormHeading With a Null Label Value is the Destroyer of Worlds (and VMs) 2009-03-24T00:00:00-07:00 <p>This afternoon I came across some rather annoying behavior in Flex 3.1.0, that I'm going to go ahead and call a bug.</p> <p>I was using a FormHeading tag at the top of my form, and I wanted it to change dynamically. So I had something like this:</p> <pre>&lt;mx:Binding source="_object.title" destination="heading.label" /&gt; ... &lt;mx:Form&gt; &lt;mx:FormHeading id="heading" /&gt; &lt;/mx:Form&gt;</pre> <p>And it works just fine as long as <em>object exists and is not null. The value is filled in nicely. But once the component has been initialized and created, if at any point in the future </em>object is set to null*, then the Flex VM throws an exception and bombs out; in fact, no other code will even execute until you restart the application. That's worse than a usual Flex exception, which typically allow you to continue doing things even after the error occurs.</p> <ul> <li>And this happens frequently in this instance, because _object is bound to a field in my Cairngorm model locator, and on another page I set the bound value to null to reset its view.</li> </ul> <p>But if instead of the FormHeading, I have this:</p> <pre>&lt;mx:Binding source="_object.title" destination="heading.text" /&gt; ... &lt;mx:FormItem label="Title"&gt; &lt;mx:Label id="heading" /&gt; &lt;/mx:FormItem&gt;</pre> <p>Then it works perfectly whether _object is set or is null. What the hell?</p> <p>Well, let's take a look at the code in <a href=""></a> that's throwing the exception:</p> <pre>private function createLabel():void { // See if we need to create our labelObj. if (_label.length &gt; 0) {</pre> <p>The exception is being thrown at the line accessing <em>label.length (</em>label is the string value of the text to display). Obviously, if _label is null then the exception gets thrown, because you can't do that.</p> <p>So how come it works in <a href=""></a>?</p> <pre>public function set text(value:String):void { // The text property can't be set to null, only to the empty string. if (!value) value = "";</pre> <p>This is exactly what FormHeading needs to do; check if the value is null, and instead of trying to access its length first, check to see if it's null. If it is, set it to an empty string, and <em>then</em> you can check its length.</p> <p>Now, I haven't yet figured out an adequate workaround. Using a Label "works," but I don't think it's as elegant as FormHeading.  I really think a nullity check of _label should be added to FormHeading.createLabel(), so this would work.</p> <p>I <em>suppose</em> this could be considered a "feature," if you want to say that a form's header/title mustn't be dynamic (which is false, I do it all the time, this is just the first time it's ever been possible for it to be null), or perhaps if a form <em>must</em> have a title (again false, not all forms have a FormHeading defined, and if the value is null it could just fall back to the non-FormHeading behavior of not showing anything).</p> <p>So I don't buy that it's a feature. I think it's a bug, and I think it'd be easy to fix. (Given that the fix is as simple as adding if (!<em>label) { </em>label = ""; } at the beginning of the method.)</p> <p>Unfortunately, my project can't wait for the next version of Flex, even assuming they decide to fix this. So I have to fix it with an inelegant, mostly unacceptable workaround.</p> <p>While it's supremely annoying that this bug exists, it's nice that Flex is open source and I can investigate what the actual problem is. If I couldn't look at the source, I'd have to assume that the problem is in <em>my</em> code and fall back to an inelegant workaround (after trying for an inordinate amount of time to find a way to get it to work correctly) without knowing why.</p> <p>So, Adobe, any chance you put that fix in soon so nobody else has to suffer this indignity? Thanks.</p> The Wells Fargo Overdraft Protection Nightmare 2009-03-22T00:00:00-07:00 <p>I used to have a bank account at Wells Fargo. When I opened it, they required me to also take a credit card along with it. Well, they didn't technically require me to; they said if I took a credit card account then both it and my bank account would be free, but if I didn't then the bank account by itself would cost $15/month or some ridiculous thing. I'm going to go with "required" me to take the credit card.</p> <p>They came up with this really brilliant system of having the credit card and checking account rely on each other: the minimum credit card payment is automatically deducted from the checking account, and the credit card is used as overdraft protection for the checking account.</p> <p>I hope you can see the nightmare scenario that is obviously possible here. Wells Fargo couldn't. I tried to explain it to them at the time, and the guy at the desk furrowed his brow, scratched his head, and said "You don't understand. It's overdraft protection! It's a good thing!"</p> <p>Well, I soon stopped using Well Fargo. Mostly because I moved to Chicago and they don't have branches here. And their web interface forgot my password, so I decided they can go stuff themselves. This was six years ago. Apparently, though, I left myself in position for the nightmare scenario. And I didn't know about it until today, when Wells Fargo called me.</p> <p>There had been a charge made on my credit card. Since I never used the card, I don't know what this could have been. Since it happened six years ago, not only do they not have a record of it, it's well past the 60 day window for reporting fraudulent charges. So Wells Fargo is not interested in the reason for this single small charge on my credit card: in their eyes, it was the first in a long string of consistent "activity."</p> <p>Because every month, the checking account automatically made the minimum payment on the credit card. That looks like activity to Wells Fargo. But I wasn't using the checking account any more; there was no money in it. You'd think that would make the payment fail and cause Wells Fargo to call me ...</p> <p>But it had overdraft protection! It's a good thing.</p> <p>So after making the minimum payment, the checking account bounced, and the overdraft was triggered ... adding the value of the credit card payment to the credit card balance, plus the overdraft charge. Each month the principal grew due to interest and the overdraft penalty, so each month the minimum payment was larger, so each month the overdraft was larger.</p> <p>I never knew any of this was happening. I never knew there was a charge in the first place. I tried to explain what this must be, what it so clearly is, to the Well Fargo person on the phone. I could hear her brow furrowing, the gears in her head clanking around, spinning for the first time in years, and finally stopping with a loud clang: "No, it's overdraft protection!"</p> <p>I was not allowed to protest. I was not allowed to file a claim of fraudulent behavior. I <em>was</em> allowed to pay them $1100 or else.</p> <p>What a disaster. Do banks do <em>anything</em> right?</p> Google Chrome Doesn't Cache DNS Correctly 2009-03-20T00:00:00-07:00 <p>A few days ago, Dreamhost moved me over to their new cluster, from the old server that had been having some problems. But this post isn't about Dreamhost; it's about Google Chrome and browsers.</p> <p>After the switchover happened, visiting my sites worked just fine in Safari and Firefox; just go to the address and the browser loaded it up just fine.</p> <p>But in Chrome, it was throwing errors. Address not found. Bad httpd_config. Nothing would go through. I left Chrome open (since there was an article I wanted to read open in one of the tabs), and the problem persisted for a few days. Finally, this morning, I restarted Chrome and it re-resolved the DNS and could actually find my sites.</p> <p>So apparently Chrome doesn't use the OS-level DNS cache, and instead does it on its own. I understand that they think they can do a better job at networking than Windows XP (which I have to use at work), but in this case it puts itself at a significant disadvantage to all the other browsers.</p> <p>I don't want to say that Chrome should just get rid of its DNS cache; they surely put significant time into it, and I'm sure they think the performance gains are significant, since they never have to go check DNS servers a second time. But it seems to me that they should be able to check whether or not someone has successfully visited a site during the current browser session (which can be long-lasting, sure, but they keep records of all of it), and if they have and when they try to visit it again and the cached DNS record doesn't find the site, that they should go out and check the DNS servers again to see if the site's address has changed.</p> <p>Obviously it's rare for a website's IP address to change during a browser session. And most people, I suppose, don't leave their browser open all the time. But given that Chrome is clearly encouraging people to use websites as web applications, and therefore to leave the browser open all the time, I think they should cover this case.</p> <p>I know it's probably just me, but this led to a couple of days worth of annoyance. And I'd rather not be subjected to that, even if the fix is copying the URLs of the articles I want to read and restarting the browser. I shouldn't have to do that.</p> Who Cares if Twitter Ever Makes Money? 2009-03-16T00:00:00-07:00 <p>Yet another "analyst" is saying that <a href="">Twitter can never make money</a>, and everyone should be afraid of that. This particular person is warning "real companies with real revenue" to stay away and not buy them.</p> <p>Frankly, I don't care if Twitter gets bought, and I don't care if they ever make money. And I also don't care if they go out of business and disappear. How difficult would it be for someone to build a new one? A small scale one, that's interoperable with a dozen other small scale replacement-Twitters?</p> <p>The answer, in case, you were wondering, is "not difficult at all." Sure it'll take a couple of months to come up with an open standard, and there'll have to be some lowest-common-denominator so it can still work via SMS (because for some reason people care about that). iPhone applications will have to be rewritten to be able to log in to multiple services. Same with desktop clients, and everything else. But adding new authentication options would be pretty trivial.</p> <p>It's bound to happen, whether Twitter runs out of money and has to go out of business, or they get bought and their new owner screws everything up, or they try to monetize it and it gets annoying for everyone.</p> <p>But, honestly, <em>who cares</em>? Nobody's that invested in Twitter that they can't jump over to something else. The only thing keeping you on Twitter is the people you're following, and that are following you. An interoperable system that allows you to have the same communications would be just as good, and would be just as trivial to sign up for.</p> <p>Honestly, I can't figure out why everyone's so worked up about how Twitter might possibly make money. It's not our problem.</p> Announcing Tiny VH 2009-03-12T00:00:00-07:00 <p>Yesterday went down (briefly, I guess). I'd been using it to shorten my links, and it was pretty frustrating that it went down. And I've read that some of these URL shortening services are planning to try to monetize their usage. Possibly with interstitial ads. I don't know about you, but I'd find that pretty infuriating.</p> <p>Therefore, I am here to announce the launch of, which is a new URL shortening service that I wrote. It's hosted at Dreamhost, and will go down whenever they do. Hopefully they move me to their new cluster soon so I can get the improved speed and reliability. Either way, it's a simple program with minimal demands, and I have no reason to try to monetize it or use the data. I just want a URL shortening service I can trust.</p> <p>And I don't trust anything more than I trust myself.</p> <p>I'm already using it for a Wordpress plugin I've written that posts a tweet whenever someone comments on your blog. I'm sure I'll come up with more uses for it. So help yourself.</p> <p>If you also want to use it automatically, you can make a simple call like this one:</p> <p></p> <p>And it'll return a string, a URL that looks like <a href=""></a> ... and then when you visit that link it'll immediately redirect to <a href=""></a>. Or whatever URL you happened to enter.</p> <p>Have fun.</p> Things I Did Now Supports Things I Haven't Done Yet But Expect To Do At Some Point 2009-02-27T00:00:00-08:00 <p>Due to overwhelming demand from exactly 100% of our users, <a href="">Things I Did</a> now has todo items.</p> <p>A Thing can either be a thing you did, or a thing you have to do. If it's a todo, just enter an @ character as the first character when you enter it, followed by a space and then whatever else you want the Thing to be (including any number of tags).</p> <p>Adding it in was pretty simple.</p> <pre>def checkForTodoItem(self):     regex = re.compile('^@')     if regex.match(self.text):         self.text =  self.text.replace('@', '', 1).strip()         return True     else:         return False</pre> <p>Note that I only want to grab 1 of the @, and only if the string starts with one. You can still have Things with @ characters in them. Just in case you wanted to do that. Which exactly 0% of our users have <em>ever</em> wanted to do. Hence, you must be crazy if you wanted to do that.</p> <p>And then when I save it I just call</p> <pre>thing.todo = thing.checkForTodoItems()</pre> <p>and the deed is done.</p> <p>I made it so the todo items show up separately from the regular things, at the top, and are a different color. The same is true even when you're viewing all the things marked with a tag, so you can tell which things you haven't done yet.</p> <p>And you can mark an item done by clicking on the "Done" link on the main /did/ page. Marking it as no longer a todo item and just a Thing you did is as simple as:</p> <pre>if thing.todo:     thing.todo = not thing.todo</pre> <p>Bam. Done.</p> <p>Things I Did just got a whole lot more useful. You don't have to wait until after you do a thing before you enter it in!</p> Flex: Make Sure to Call Super in Item Renderer Override Methods 2009-02-25T00:00:00-08:00 <p>Today I came across a bug where I was using an item renderer in a data grid and the row didn't highlight when you mouse over it and wouldn't get selected when you clicked on it (ie, moused over or clicked on the column, not the entire row). The item renderer consisted of a Text component (so I could use htmlText) inside of a VBox (so I could use horizontalScrollPolicy="off").</p> <p>At first, it was tough to see what the problem could have been. The item renderer is very simple, and I'm using more complicated item renderers elsewhere that don't have this problem. I thought it might be a symptom of using Text, but switching it to a Label (and losing my HTML rendering) did not change anything.</p> <p>That's when I noticed that my data setter override didn't look quite right:</p> <pre>public override function set data(value:Object):void {     _text.htmlText = value.content; }</pre> <p>Well, that's not enough! I was <em>this close</em> to trying to hack something together with mouseover and click event handlers, when I realized I needed to do this:</p> <pre>public override function set data(value:Object):void { = value;     _text.htmlText = value.content; }</pre> <p>Apparently something in the super class's data setter does something vitally important, and you really shouldn't skip that. If you do, weird stuff can happen, including data grid columns that don't behave right on mouseover or click.</p> <p>Lesson learned.</p> Things I Did Gets Tags 2009-02-22T00:00:00-08:00 <p>Today I put together a quick new feature update for <a href="">Things I Did</a>. I've been primarily using it at work, to record what I do during the day; but if I wanted to record something I did outside of work it kind of gets lost amongst all the other things.</p> <p>Obviously, the solution is a tags/labels concept, where each thing can have an arbitrary number of tags. I'd been thinking about it for a while, and hadn't come up with a good way to do it. I was leaning toward a system where you manually add tags, then select them from a list (which checkboxes) when creating a new thing. At best, even in my imagination, it was really cumbersome.</p> <p>Since the behind-the-scenes implementation should be the same regardless of how the interface is constructed, I decided to go ahead with starting it. I added a simple Tag model and put a ManyToManyField on the Thing class.</p> <p>While I was playing with getting them connected, I was struck by a much simpler, more text-based interface for tags: #hashtags right there in the thing's text.</p> <p>I needed a way to extract them from the text, so I added a method to the Thing model:</p> <pre>def extractHashTags(self):     hashRegex = re.compile('#([\w\-\_]+)')     tags = hashRegex.findall(self.text)     for tag in tags:         self.text = self.text.replace('#%s' % tag, '').strip()     return tags</pre> <p>I'm using a regex that looks for a # followed by letters, hyphens, and underscores, and use findall() to get all of them. (Simply using match() will only get the first one.) I then loop through each tag that was found and remove it from the text, because I don't want to save the tagname as part of the Thing.</p> <p>But I'm not out of the woods yet, I still need to add something to the add() view to manage the tags on a new Thing. So I replaced the simple line with all this:</p> <pre>thing = Thing(user=user, text=text) tagTexts = thing.extractHashTags() for tagText in tagTexts:     tags = Tag.objects.filter(user=user).filter(text=tagText)     if tags:         tag = tags[0]     else:         tag = Tag(user=user, text=tagText)     thing.tags.add(tag)</pre> <p>After creating the Thing with the full text (including #hashtags), I extract them and get a list of them. Before I can add the tags (if there are any), I have to save the Thing; ManyToManyField relationships don't work without primary keys. I loop through each of the #hashtags I found and check if they already exist -- if not, I have to create them. Then I add each one to the Thing's list of tags, which automatically saves.</p> <p>The release process was a little cumbersome; I had to keep track of the newly created tables in apps that were already installed in the file. It'd be nice if Django had some concept of Rails migrations.</p> <p>But it's working pretty nicely now. With a bit of use this week I'll be able to tell if I like the new feature.</p> Flex: Selecting the Second Tab of a TabNavigator Without Flashing the First 2009-02-19T00:00:00-08:00 <p>Flex's TabNavigator is a great thing. In normal situations, it's easy enough to use:</p> <p><code>&lt;mx:TabNavigator id="tabNavigator"&gt; &nbsp;&nbsp;&nbsp;&nbsp;&lt;vh:ComponentZero id="componentZero" label="Zero" /&gt; &nbsp;&nbsp;&nbsp;&nbsp;&lt;vh:ComponentOne id="componentOne" label="One" /&gt; &nbsp;&nbsp;&nbsp;&nbsp;&lt;vh:ComponentTwo id="componentTwo" label="Two" /&gt; &lt;/mx:TabNavigator&gt;</code></p> <p>It'll load up ComponentZero first, and there will be three tabs at the top of the TabNavigator that let you switch between them. Awesome. Except sometimes you find yourself in a situation where you want to load the <em>second</em> tab instead of the first, but you want the order to stay the same.</p> <p>I was in that situation today. At first it seems like you can just do something simple:</p> <p><code>private function _onCreationComplete(event:Event):void { &nbsp;&nbsp;&nbsp;&nbsp;tabNavigator.selectedChild = componentOne; }</code></p> <p>That'll jump you over to the second tab almost right away. Problem solved, right? Wrong! When it builds the page, it flashes the first tab for a fraction of a second before it moves you over to the second tab. That doesn't look so great. And in MXML, there's really nothing you can do about it.</p> <p>The only way I could figure out to get around this is to build out the children of the TabNavigator in ActionScript, building the second tab first, then inserting the first tab in front of it. It's kind of cumbersome and isn't as smooth as just doing it with MXML, but it gets you what you want. If, that is, you want what I want. Which you'd better.</p> <p><code>private var componentZero:ComponentZero; private var componentOne:ComponentOne; private var componentTwo:ComponentTwo;</code></p> <p><code>private function _onCreationComplete(event:Event):void { &nbsp;&nbsp;&nbsp;&nbsp;componentOne = new ComponentOne(); &nbsp;&nbsp;&nbsp;&nbsp;componentOne.label = "One"; &nbsp;&nbsp;&nbsp;&nbsp;tabNavigator.addChild(componentOne);</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;componentZero = new ComponentZero(); &nbsp;&nbsp;&nbsp;&nbsp;componentZero.label = "Zero"; &nbsp;&nbsp;&nbsp;&nbsp;tabNavigator.addChildAt(componentZero, 0);</code></p> <p><code>&nbsp;&nbsp;&nbsp;&nbsp;componentTwo = new ComponentTwo(); &nbsp;&nbsp;&nbsp;&nbsp;componentTwo.label = "Two"; &nbsp;&nbsp;&nbsp;&nbsp;tabNavigator.addChild(componentTwo); }</code></p> <p><code>...</code></p> <p><code>&lt;mx:TabNavigator id="tabNavigator" /&gt;</code></p> <p>Note that I had to create the second tab first and add it to the TabNavigator, then I had to use addChildAt() to add the tab that I want to appear in front of it.</p> <p>If you're ever stuck in where I was, hopefully this helps. If you've got a better way to do this, I'm all ears.</p> PHP Memcached Preloading Cache 2009-02-15T00:00:00-08:00 <p>I'm working on a project for a high-read, low-write environment, and one of the things we want to do is keep a database as far away from the production webservers as possible.</p> <p>So I want to generate content somewhere else before uploading it to the webservers to be displayed. But we tried just generating all the static content, and it's not going to work; too much duplicated data and wasted space, and too many separate files.</p> <p>Instead, I made a flat file with all the necessary information in it, with each tuple contained to its own line and data fields separated by | characters. Given one of two keys, I can scan through the file and find the tuple it belongs to. Once I've gotten the data from the file, I stick them into the templates on the webserver as opposed to generating it all at once.</p> <p>But it seems wasteful to scan the file <em>every time</em> I need to look up a tuple. This is a high-read environment, and I don't want to spend all our time spinning through a flat file looking up the tuples. So I need memcached. On Ubuntu, it's really easy to set it up:</p> <pre>sudo aptitude install php5-cli sudo aptitude install php5-memcache sudo aptitude install memcached memcached -d -m 1024 -l -p 11211 sudo /etc/init.d/apache2 restart</pre> <p>At this point you can run PHP from the command line, access memcached from it, and a memcached server is running and is only accessible from this server, and Apache has been restarted so it can load up the new PHP extension.</p> <p>Since I upload the new datafile daily, I'm going to need to flush the cache and preload some tuples into the cache -- for our higher-volume customers it'd be nice if we <em>never</em> need to look them up based on a customer visit. This is why we needed to run PHP from the command line; we have a script that does this.</p> <pre>$memcache = new Memcache; $memcache-&gt;connect('localhost', 11211);</pre> <p>Now that we've connected to the memcached server, we want to flush it out; because of a limitation in PHP's memcache plugin, we need to wait one second after flushing before entering anything new into the cache.</p> <pre>$memcache-&gt;flush(); $time = time() + 1; while (time() &lt; $time) { /* spin */ }</pre> <p>Now we're ready to look up some tuples and cache them. Here's where we load the keys we want to look up:</p> <pre>$ini = parse_ini_file(PRELOAD_FILENAME);</pre> <p>What? Oh right ... here's the file with the codes we want to load:</p> <pre>codes[] = dffo1000 codes[] = dffo50000 codes[] = dffo12121 ...</pre> <p>I'll need to generate this list of most-frequently-used codes on the server and upload it to the webserver every time I generate a new datafile, but it shouldn't ever get that big and it still beats the trash out of generating all the content statically.</p> <p>In PHP's ini files, putting [] at the end of the variable name means it's an array. Now we can loop through these and get each one like so:</p> <pre>foreach ($ini['codes'] as $code) { /* lookup and cache! */ }</pre> <p>I don't want to get into the actual lookups right now, but the caching looks like it's working great. And that was really the number one goal here.</p> Parsing a Flat File Without Loading the Whole Thing 2009-02-15T00:00:00-08:00 <p>Okay, I want to talk a little bit about parsing a flat file in PHP. Usually when you see something about it online the recommendation for doing it in PHP is to use file() to get the contents of the file in an array, where each line is an element of the array. Then use explode() on each line. Well, as the filesize grows, that method uses up more and more memory until your script starts to blow up.</p> <p>There must be a better way!</p> <p>So here, I'm assuming that each line has a single pipe (|) as a delimeter. I want to scan through the file and only grab one line at a time, tossing each line after it's been processed -- and stopping in the middle if we want to. The fscanf() method allows you to just that, using a regex to break up your text however you want to.</p> <p>Well, let's just say I want to take the line that looks like "field1|field2|field3|field4" and turn it into an array. The fscanf() method's regex allows you to break up the matched string into an array using %[] for each match. Here's the call I use:</p> <pre>fscanf($file, "%[^\|]|%[^\|]|%[^\|]|%[^\|\n]\n")</pre> <p>Note that I pass in a file handle received from fopen(), and that I have a \n at the end; without that, fscanf() doesn't know you want to go line by line. The fourth field adds a \n into its match -- that way, we don't have to trim() the last field when we're done.</p> <p>It now returns an array that looks like [ field1, field2, field3, field4 ]. Which is great. But it does that for every single line in the file, and I need to find out if this is the one I'm looking for before we move along to the next one. In my application, I want to be able to search by any of two fields, so that's what I'm going to demonstrate here.</p> <p>I don't want to repeat my parse() method, so I pass in a $parseType value in addition to the $key I'm searching for. $parseType will tell parse() which field to use and how to do the comparison.</p> <p>Here are the methods that I'm using to compare by each field.</p> <pre>private function field1Parser($line, $key) {     $field = $line[0];     $field = preg_replace('/\*/', '.*', $field);     if (preg_match("/^{$field}$/", $key)) {         return new MyCrazyObject($line);     } else {         return null;     } } private function field2Parser($line, $key) {     $field = $line[1];     if ($field == $key) {         return new MyCrazyObject($line);     } else {         return null;     } }</pre> <p>Note here that when we're searching on field1, we replace all the * characters with .* and do a preg_match() on it to check if it's the line we want, whereas on field2 we just compare directly. The latter is much much faster, but there are some cases where we simply need to do the former in order for it to work. Since the parsing method can differ, I need a separate method for each one rather than simply defining which array index to look for.</p> <p>Here's a simplified version of my parse() method without error checking or caching.</p> <pre>private function parse($parseType, $key) {     $file = fopen ($this-&gt;filename, 'r');     while ($line = fscanf($file, "%[^\|]|%[^\|]|%[^\|]|%[^\|\n]\n")) {         if ($parseType == MyCrazyParser::FIELD1) {             $obj = $this-&gt;field1Parser($line, $key);         } else if ($parseType == MyCrazyParser::FIELD2) {             $obj = $this-&gt;field2Parser($line, $key);         } else {             $obj = null;         }         if ($obj != null) {             fclose($file);             return $obj;         }     }     fclose($file);     return null; }</pre> <p>Obviously you'll need error checking to validate that the parse type is acceptable, that the file exists, etc. And you're going to want to check if the given key is found in the cache, and cache it when it's found; and also cache that the key was NOT found, so that later on we don't have to read through the entire file every time we're looking for something that isn't found. I took all that stuff out for time/space reasons and because it's simple and not relevant to the actual discussion of parsing. (And I tried saving this thing once and Wordpress bombed out on me and I've had to write the second half of this article including parse() twice, which is extremely frustrating.)</p> <p>The field1Parser() is 10x slower than the field2Parser(), because of the regex replacing and matching versus simple equality. Which makes it especially necessary to cache. I decided to cache by both keys when an object is found, so that I can come back later and grab it by the other key if I want and not have to worry about searching the file all over again. But we're not really talking about caching here.</p> <p>I believe using fscanf() in this manner is much better than using the crappy old file()/explode() technique. But I'm not satisfied that this is the best. I'd like to be able to use closures/anonymous functions instead of calling named methods for the different parse types; in fact, the whole parse type technique (requiring a class-level constant for each valid type) seems crufty to me.</p> <p>If anyone has seen any techniques that are smoother, simpler, or faster, I'd love to know about it.</p> Things I Did; Django on Dreamhost 2009-01-31T00:00:00-08:00 <p>A while back, I wrote an article that said <a href="">goodbye to cloud computing</a>, at least for programs I write myself. I found that I couldn't trust the combination of AppJet and Facebook to work reliably enough to keep my application running, and I had no control to fix it if there were problems.</p> <p>Well, the application I'd written for the AppJet/Facebook "platform" was a simple little thing called "Things I Did" -- the idea being to keep track of the things you've done over the course of a day, so that at the end of it you know what you did. I find I always have trouble remembering what I did at work when it comes time for the daily or weekly recap, so this would help me remember. And it'd be convenient to get a weekly report so I can even remember what I'd done for the week to help me update my status reports.</p> <p>Well, I've rewritten the application such that I can host it myself. I prefer to write in Python, so naturally I went with a little Django application. The tough part was actually deploying it on Dreamhost. Fortunately, Jeff Croft has a great little tutorial on <a href="">actually getting Django working on Dreamhost</a>. So after about an hour of setting it up, re-setting it up after something didn't work, and tweaking my application to work in the new environment, I've got the thing running. I really thing it should be easier than this to get Django applications running on Dreamhost -- and I don't know that you could have multiple Django applications running on one Dreamhost account. (You have to add a pointer to your application's in your .bash_profile ... which means to me that you can run a max of one application on one account, even though a single account can host many sites.) Ultimately, this is going to be a dealbreaker for me with Django on Dreamhost.</p> <p>But, for now, I have <a href="">Things I Did</a> up and running. I'll keep it up and running, until at some point in the future I rewrite it with something other than Django that's more suitable for Dreamhost, or move to another provider that's more amenable to Django. I definitely want to be able to write more than one application hosted on my account.</p> <p>So check out <a href="">Things I Did</a> if you have trouble remembering what you did yesterday!</p> ESPN Fantasy Football Analyzer 2009-01-16T00:00:00-08:00 <p>I like to play fantasy football, but I think ESPN could offer more data.</p> <p>Particularly, I want to know how bad I am at it, as well as a way to settle arguments about whose team is the best in the league. Typically, the team with the best record claims he's the best; however, everyone else notices that the player with the best record often has fewer points against than the other team. Also, some fantasy football players (and by some, I mean <em>all</em>) set their record improperly and players score big points on their bench.</p> <p>If you have a bunch of players who score well for your bench, it means your team is good, just that <em>you</em> aren't very good at setting your roster.</p> <p>So I wanted a way to calculate <em>optimum points</em> and <em>optimum record</em> to help settle these debates. The concept of optimum points is that it's your score if you'd set your starting roster optimally -- and started your players such that you get the most possible points. Your optimum record is what your record would be if you and all your opponents had played optimally every week. The team with the most optimum points is probably the best team; the team with the best optimum record is the one that should have won the league. Note that this is <em>not</em> necessarily the same team, given that there's still luck involved in who you play.</p> <p>While the team with the fewest optimum points is the worst team in the league, the team with the largest gap between actual and optimum points has the worst owner. If your bench players keep scoring big, you're probably not very good at this.</p> <p>So I came up with my <a href="">ESPN Fantasy Football Analyzer</a> to calculate this stuff. That's the GitHub repository, so go check it out and use it to analyze your fantasy team if you're curious about how bad you are at fantasy football.</p> <p>I originally wanted to make it so it crawled ESPN's league page and analyzed all the games automatically -- the problem is that ESPN requires that you log in to see the league's schedule and boxscores, and I couldn't get Scrapy to successfully pass the login cookies to ESPN to authenticate and get the pages. So I just downloaded the quick boxscore from each game manually and put them into a directory structure that looks like this:</p> <pre>YEAR/ WEEKS/ GAMES</pre> <p>For example:</p> <pre>2008/ 1/ 1 2 3 2/ 1 2 3</pre> <p>etc...</p> <p>I go through each specified game, and use a series of regexes to pull out the data I need. I don't need to bore you with the details, but I needed team names, player names/ids, player positions, the "slot" the player played in that week, player points -- for all players on each team, whether they were in the starting lineup or the bench.</p> <p>As I analyze each game, I keep track of the two teams involved, the score lines of each player on their team, and calculate the actual and optimum points for each team and come up with the actual and optimum winner. I add each GameScore to the Season.</p> <p>Once I have all the GameScores, I analyze the entire season and pull out the teams and players from each game. It's at that point that I can get a definitive list of all the teams in the league and all the players who were on a roster at any point during the season.</p> <p>I could try to explain the whole thing right now, but I think I'll either wait till another post or just skip it. I encourage you to give it a shot so you can finally know just how much better you should have done.</p> <p>(It turns out my team wasn't as good as I thought -- my optimum record was actually worse than my actual record, and several teams had higher optimum points and larger gaps between actual and optimum.)</p> <p>Oh, and another thing it does is record which players scored above their season average against your team. That's just so you know which players hate you.</p> <p>For example, Deangelo Williams played me twice, and scored 32 and 34 points in those games. Deangelo Williams hates me and my fantasy football team.</p> <p>Enjoy!</p> Damn the Cloud 2009-01-10T00:00:00-08:00 <p>Hosting your applications in the cloud. It sounds so great. You just develop it, and then release it somewhere out in the internet, where it can run forever without you having to worry about it. It's cheaper, faster, and you don't have to maintain or pay for any systems. It can even scale rapidly to new users and usage as needed! Amazing!</p> <p>Except that you cede control of your application to someone else. You're at their whim as to whether your application runs or not. If their systems have a problem, you're stuck. Not only are you unable to get your program running again, you're not even able to pull your code and data out and run it elsewhere -- you're locked into their system. You just have to trust them, that a) they will never raise their prices, b) they will never have an outage, and c) they will never go out of business and/or disappear. Do you trust <em>anyone</em> to meet those requirements?</p> <p>As an example, I recently developed an application using AppJet, and released it on Facebook. Seemed like a good idea; it was just a simple application, and it was something I wanted to use myself and get out the door as quickly as possible without having to do any administrative rigamarole, like taking the time to set up Django on Dreamhost, or something.</p> <p>Check out the <a href="">code for the application</a>, or <a href="">go use it at Facebook</a>. The basic idea is that you write down all the things you do over the course of the day, and then later you can go back and check what you did, to help you remember. I wanted it because at the end of the week my boss needs to update our project status and asks "What did you do this week?" And I can never remember. A week is a long time.</p> <p>So it took me just an afternoon to create the application using AppJet's nice Javascript environment, and even less time to link it up to Facebook. It worked well enough, despite a couple of initial problems. First, their query language is pretty limited: for example, you can't use less-than or greater-than on dates (or anything else), so if I wanted to set up a date-based search to narrow down what I did by date, well, I can't. The other initial problem was that I didn't know you had to get the Facebook user to "allow" your application just to get their user id. The documentation said you needed to do that to get access to their personal information, but did not specify that the user id is considered personal information. I didn't discover that little bug until my application got its second user and I could see all their entries ... and discovered that they could see all of mine. It was the work of a moment to fix it, but it was a little embarrassing.</p> <p>But once I got those initial problems ironed out, I had my application running "in the cloud." It was nice. I didn't have to worry about how it was running, because someone else was taking care of it. In fact, I <em>couldn't worry</em> about it, because I had absolutely no control. Which wouldn't have been a problem, except there soon started to be outages.</p> <p>After a couple of weeks, Facebook started regularly having problems accessing AppJet. I'd log into AppJet to see what was up, and sometimes they'd be down, sometimes they'd be slow, sometimes they'd be working just fine and the inability to communicate just made no sense. The important thing is that there was nothing I could do about it.</p> <p>When there's an outage on Dreamhost, I can shoot off an email to their support people asking what's going on, and I very quickly get a response explaining what the situation was, an estimate as to how long it'll take to fix it, and an apology for the problem. If I were paying them more money, I could probably get even better service.</p> <p>But AppJet and Facebook are "free." While that means free of charge, they're also free of accountability, free of customer service.</p> <p>It's become extremely irritating. The outages happen more than daily, and my application has become unusable. I'm back to having trouble remembering what I did at the end of the week.</p> <p>So I'm going back to writing outside the cloud. I'll write Django apps and put them on my hosted account. I will not release applications on these cloud servers, as long as I have a choice. Hopefully people have a choice for a while longer.</p> New Leveling Process 2008-12-25T00:00:00-08:00 <p>So this morning, I came up with a considerably better way to define the levels for my number grid game. (Yes, I did that on Christmas morning. What?)</p> <p>Instead of the ridiculous if-statement I put together last night, which would have been more than a little cumbersome to maintain, I now have a simple list of point values that define the points required for each level. It's not the algorithm I was hoping for (I guess I have to do some more thinking), but at least I can add more levels simply.</p> <p>Here's the array with the levels defined:</p> <pre>private static const LEVELS:Array = [ 0, 200, 600, 1000, 1500, 2500, 3800, 5500, ];</pre> <p>The way it works, the index of the array is the level, and the value at each index is the points required to graduate from that level. Since you start at level 1, the first value in the array (index zero) is useless.</p> <p>To check if it's time to level up, I've replaced the ridiculous if statement with this:</p> <pre>private function get _canLevelUp():Boolean { return (_points &gt; LEVELS[_level]); }</pre> <p>Awesome. I think that part, at least, is pretty nice.</p> <p>As before, <a href="">see the code at GitHub</a> or just go <a href="">play the game here</a>.</p> A Simple Edutainment Game: It's Math Bejeweled! 2008-12-25T00:00:00-08:00 <p>I read an article earlier today, <a href="">exhorting people to write "edutainment" software</a>; there were multiple complaints in the article: teaching is hard so teachers should be classroom managers who turn to an authoritative knowledge database for answers to their students' questions, children are addicted to video games so there should be video games that teach them things instead of waste their time, and emerging markets have less money so there should be software for cheap cell phones from which people can somehow learn.</p> <p>I agree with the idea that there should be an authoritative knowledge database, so that teachers can teach the same curriculum nationwide. But solving the "some students already know what you're trying to teach them while others are unwilling/unable to learn the simplest material, and the former are the ones you should push while the later are the ones that require the most time and attention" problem is much more complicated than just some software, I think. And software for cheap cellphones that teaches physics to people in third world countries while they're working in rice paddies, while noble, is neither here nor there.</p> <blockquote>And that brings us to the humongous entrepreneurial opportunity that immersive, engaging educational software presents. Why is it that children are addicted to games like <em>World of Warcraft</em>? And why have we not yet created educational software and games that are equally addictive?</blockquote> <p>Some kind of MMORPG that teaches you math would sure be interesting ... except that those games are typically designed as a fantasy escape world, where the rules of reality don't apply. So those types of games wouldn't work that well for teaching, really. (History maybe. Not math or physics.) But I think the author was using World of Warcraft as an example, and that what she really meant was "any video game at all." In my opinion, Bejeweled would be a better example.</p> <p>It's the kind of game people waste time with, and can jump in and out at their leisure. They can play for three minutes or three hours. If there were a version of Bejeweled that made you perform simple arithmetic rather than match colors together, that'd be the kind of game I could actually imagine some kid playing and learning from. (Adding numbers isn't much more difficult than matching colors, really.)</p> <p>So this afternoon, after I got off work, I threw such a simple game together. I wrote it Flex, because that's what I've mainly been using at work, and it's pretty fun. Also, it's pretty much ideal for this sort of thing.</p> <p>The idea is that there's a "target number" that you have to hit by adding numbers together from a grid. You click a number in the grid to select it, and keep selecting numbers until you've hit the target. You get points for the size of the selection (ie, the more numbers it takes to add up to the target, the more points you get), as well as the amount of time it takes to do it (the faster the better). The target number changes when you reach a certain amount of points.</p> <p>The first thing I needed was a clickable Cell that shows a number and can either be selected or not. That's in, and here's the important bits:</p> <pre>public function Cell(number:Number=1) { super(); this.number = number; this.toggle = true; this.setStyle("fontSize", 14); this.width = 50; this.addEventListener(MouseEvent.CLICK, _clickHandler); }</pre> <p>That sets up the Cell and gets it ready to be clickable. (It extends LinkButton, so it's ready to be clicked. We set up how it looks and then connect an event listener to handle toggling between selected and unselected. Another important method:</p> <pre>public override function set selected(value:Boolean):void { super.selected = value; _selected = value; if (this.selected) { this.setStyle("color", "red"); this.setStyle("textRollOverColor", "red"); } else { this.setStyle("color", "black"); this.setStyle("textRollOverColor", "black"); } }</pre> <p>We need to override the "selected" setter to flip between the selected an unselected states -- for now I just have it so the number is red when it's selected and black when it isn't. It works well enough for me, but later on I'll have to figure something out to make it look better.</p> <p>That's just the cells in the grid. The next thing we need is the grid itself. So I create NumberGridView.mxml and toss an &lt;mx:Grid /&gt; element in there. Then I want to dynamically build the grid full of Cells. I do that when the component has initialized:</p> <pre>private function _onInitialize():void { _generateTargetNumber(); for (var i:int=0; i &lt; GRID_SIZE; i++) { _cells.push([]); var gridRow:GridRow = new GridRow(); grid.addChild(gridRow); for (var j:int=0; j &lt; GRID_SIZE; j++) { var cell:Cell = new Cell(_randomCellValue); var gridItem:GridItem = new GridItem(); gridItem.addChild(cell); gridRow.addChild(gridItem); _cells[i][j] = cell; cell.addEventListener(MouseEvent.CLICK, _cellClickHandler); } } _resetTimer(); }</pre> <p>In that method, I generate a new target value, start the timer, and build the grid of cells. I first have to create a GridRow, then fill it with GridItems; each GridItem has a Cell inside it, and each Cell is seeded with a random number. Also, I add another event listener to each cell -- I want to handle some stuff at the grid level when a cell is clicked. Namely, I need to check if the target has been met and act accordingly:</p> <pre>private function _cellClickHandler(event:Event):void { // get the sum of the selected cells var sum:Number = _sumSelection; // check if the target has been met if (sum == _targetValue) { // add points to the total based on the selection _points += _calculatePoints(); // check if they can level up if (_canLevelUp) { _level += 1; _generateTargetNumber(); } // reset the start time _resetTimer(); // generate new numbers for each of the cells var cells:Array = _selectedCells; for each (var cell:Cell in cells) { cell.number = _randomCellValue; cell.selected = false; } } }</pre> <p>I just get the sum of the selected cells, and check if it matches the target. If it does, then I calculate how many points they got for that selection, see if they leveled up, generate new numbers for those selected cells, and reset the timer. No problemo.</p> <p>When I want to get the currently selected cells, I need loop through my matrix of cells and use the public getter on the Cell object:</p> <pre>private function get _selectedCells():Array { var cells:Array = []; for (var i:int=0; i &lt; GRID_SIZE; i++) { for (var j:int=0; j &lt; GRID_SIZE; j++) { var cell:Cell = _cells[i][j]; if (cell.selected) { cells.push(cell); } } } return cells; }</pre> <p>And to generate new target values and cell values, I use the Math.random() method:</p> <pre>private function _generateTargetNumber():void { var minTarget:Number = 5 * _level; var maxTarget:Number = 10 * _level; var random:Number = Math.random(); var target:Number = Math.round(random * (maxTarget - minTarget)) + minTarget; _targetValue = target; }</pre> <pre>private function get _randomCellValue():Number { var random:Number = Math.random(); var value:Number = Math.round(random * (_targetValue - 1)) + 1; return value; } </pre> <p>The target value is based on your current level (the higher your level, the bigger the numbers get), and the cell values are based on the target value. Obviously, we don't want cell values to be higher than the target, that'd be pretty useless.</p> <p>When I want to calculate the points, I use the current target value, the number of cells currently selected, and the amount of time for the selection.</p> <pre>private function _calculatePoints():Number { var currentTime:Date = new Date(); var cells:Array = _selectedCells; var seconds:Number = ((currentTime.getTime() - _startTime.getTime()) / 1000); var points:Number = (cells.length * _level * 5); points += _targetValue; if (seconds &lt; 1) { points *= 3; } else if (seconds &lt; 2) { points *= 2; } else if (seconds &lt; 5) { points *= 1.5; } else if (seconds &gt; 10) { points *= 0.33; } points = Math.round(points); return points; }</pre> <p>The faster you hit the target, the more points you get. But the most important thing is getting a long chain; that adds up points the fastest. The reason for that is that I don't want people just hitting the cell with the same target ... but at the same time if they try to chain together a bunch of 1's, it'll take a while and then they won't have any 1's left to fill out any selections when they need it.</p> <p>One problem I have is the leveling up:</p> <pre>private function get _canLevelUp():Boolean { if (_level == 1) { return (_points &gt; 500); } else if (_level == 2) { return (_points &gt; 2500); } else { return false; } }</pre> <p>For now, obviously, I'm only handling two levels. I'd like to come up with a simple algorithm to define the level for any amount of points, rather than having to define the points for each level. Any ideas? Bear in mind that the number of points people can score increases linearly with their level (the points per selection increases with the target, but with higher targets it takes longer to hit the target).</p> <p>Obviously, I didn't show all the code here. And it's still a work in progress. If you want to see all the code or contribute to it and help me out, <a href="">go check it out at GitHub</a>. Or if you just want to play it and see what it's like, <a href="">go play</a>!</p> <p>I do think these simple games are the ones that could get kids addicted and get them to play and possibly learn. I don't think the goal should be to try to get them to play some ridiculous MMORPG or shooter game where they somehow learn something. This post has gone on way too long, so I'll wrap it up. Hopefully someone found it useful in some way. And also hopefully my project and work gets wrapped up and released soon -- it's actually a large program, unlike this and most of my side projects. It gets fun when projects get really big.</p> Goodbye 2008-12-24T00:00:00-08:00 <p>Another blog is born.</p> <p>I've gotten fed up with trying to use Blogger. I've been posting there, on and off, for a couple of years now. And this week, I tried to go back in and post something, and they were having technical problems that prevented me from being able to create a new post.</p> <p>So I thought, what the hell am I doing? I'll just do this on my own. And that's where we are.</p> <p>I'm going to use this one to post my thoughts and whatever I feel like ... and this will also be where I post anything about programming, now that I'm no longer using the Sean Code blog.</p> <p>Hopefully I'll also be able to do a better job of posting regularly.</p>