Roy Ashbrook

Yes, Heroku made me sad today. Heroku!? What?! I can hear you saying it! =P Fortunately I was just coming down off of a fun trip after refreshing myself on the Codeacademy ruby course. Really nothing there if you already use ruby quite a bit, but I haven’t written any ruby in a couple years and it’s always nice to remember how pleasant that language is to work in. That ruby high is all that kept me plugging away at putting dex’s evil twin ruby brother ‘fwad’ out on Heroku. But first… how did Heroku make me sad? Well FIRST-first, a little backstory.

Lately I have been doing a lot of research into cloud providers and looking at various hosted solutions for all sorts of things. So fiddling with these two services over the last couple of days gave me some time to both play with ruby and play with Heroku and AppHarbor, two pretty popular application hosting/cloud services. I originally had the idea used for dex in ruby. I wrote a local client in ruby that did everything except stream the data in a response. I say “everything” but it should be in quotes the first time as well because I just wrote something simple that would download a file and write it to disk. On a whim, I also thought “hey, why not have it gzip the data as well?” So that’s what I wrote. In fact, here is the local client in all of it’s glory:

require "open-uri"
require "zlib"
url = 'http://somesite.with/an/exe/todownload/somefile.exe'
gzf =  'c:\temp\somefile.exe.gz'
open(url) { |rf| #open remote file
  File.open(gzf, 'w+b') { |lf| #open local file/stream
    Zlib::GzipWriter.open(lf) { |gz| #open gzipper on local file/stream
      gz.write(rf.read) #write the compressed data
    }
  }
}

Not too glorious, eh? Well it was meant to be really really simple. I think I spent like 60 minutes getting ruby installed and writing this little dealio. It worked fine and thus began my campaign to somehow promote it, as a web app, to Heroku. I already had an account from when they were in beta, how hard could it be? Well, apparently for someone who works primarily on windows, without git, and without normal ruby DEV tools installed, it’s a bit of a pain. Now, I will say that it isn’t really Heroku’s fault that I was somewhat ignorant of what I wanted at the time. But in my defense the first thing I did was try and get a simple hello world site setup as described in their Getting Started with Ruby on Heroku tutorial here: https://devcenter.heroku.com/articles/ruby. Now, I’m not going to rehash my entire silly experience. But I will say that I tried to follow the directions and I still ended up having to piecemeal my solution together to get the initial Sinatra application to work. The main culprit in my whole story was this snip from that page:

Declare process types with Procfile

Use a Procfile, a text file in the root directory of your application, to explicitly declare what command should be executed to start a web dyno. In this case, you simply need to execute the web.rb using Ruby.

Here’s a Procfile for the sample app we’ve been working on:

web: bundle exec ruby web.rb -p $PORT

If you’re instead deploying a straight Rack app, here’s a Procfile that can execute your config.ru:

web: bundle exec rackup config.ru -p $PORT

Somehow I missed the top one and put the second one in. So my app kept crashing and I had to become more familiar with debugging stupid self inflicted errors on Heroku than I wanted to. Anyway, once I figured all that out and all the other silly stuff, I ended up uploading a straight rack application. I found this link (http://rubylearning.com/blog/a-quick-introduction-to-rack/) to be very helpful. So this process took me on and off almost a day to figure out and get the code written/pushed/etc. I went with rack because by the time I was done with this I had already long since rewritten the app in C# (after downloading and installing the vs. web express as I didn’t have MS web DEV tools installed, only Win/Console App tools) and deployed it to AppHarbor. that includes setting up CodePlex, etc., etc. Since I hadn’t gotten my hello world app up, I hadn’t finished the ruby version, so I wrote the C# version first really. I say that only because I can’t say I ‘ported’ the ruby code. So back to why I went with rack, it was just because I could write all the code in one page and keep it really simple.

So what mad me sad is it was so much fun to write my little ruby script and then it was so much more of a pain to get it up to Heroku than I thought it would be. I realize that it was not Heroku’s fault really, but maybe they should have an even MORE handholding tutorial =P I felt like I had to click on 100 different links to go get a tutorial on this thing or that thing. really I would have been happy to just have them generate the app in their tutorial and then let me do a git from some prebuilt repo and I could edit it from there. I’m sure not everyone wants that, but it really would have helped me out today.

So here, finally, is FWAD( firewall avoidance downloader! =D )! I haven’t decided if I like the pronunciation ‘EF’-‘WAD’ or ‘fWAD’ better yet.

Anyway, the entire contents of the app is a Gemfile with:

source 'http://rubygems.org'
gem 'rack'

And a config.ru with:

require 'open-uri'
require 'zlib'

run lambda {|s|
    
    headers = {}
    response_body = ''
    path = s['PATH_INFO'] 
    path += '?' + s['QUERY_STRING'] if s['QUERY_STRING'] != ""
    key = 'YaTl7a4akBMefeCZ'
    op = path[1,16]
    
    begin
        if (op == key || op == key.reverse)
        
            url = path[18,path.length-18] + '.exe'
            response_body = open(url,'rb'){|i|i.read}
            
            if (op == key)
                ct = 'application/octet-stream'
                cd = 'attachment; filename=renameme.txt'
            else
                ct = 'application/x-gzip'
                cd = 'attachment; filename=file.gz'
                bn = File.basename(url)
                bn = bn.include?('?') ? 'renameme.txt' : bn
                output = StringIO.new
                gz = Zlib::GzipWriter.new(output)
                gz.orig_name = bn
                gz.write(response_body)
                gz.close
                response_body = output.string
            end
            
            headers["Content-Type"] = ct
            headers["Content-Disposition"] = cd
            
        else
            response_body = 'no dude.... just... no.'
        end
    rescue
        response_body = 'no dude.... just... no!'
    end

    headers['Content-Type'] ||= 'text/plain'
    headers['Content-Length'] = response_body.length.to_s

    [200, headers, [response_body]] #we lie and say we're always ok =D

}

Note: You do still need a procfile and a Gemfile.lock to push up to Heroku so it will work. It works similar to dex however, since I could intercept the call before it went anywhere, I went ahead and ghetto handled my own routes. I used the same key as a dex for uncompressed download to txt and I set the reverse of that key to be the route for compression. I also kinda went cheap on embedding the filename in the zip. If someone sends a filename that is in the querystring, I piecemeal that in earlier in the code, but File.basename in ruby will see a full filename as something like “filedownloader.php?myfile.exe”. So I just avoid that whole mess and if there is a ‘?’ (meaning a querystring) I just send gzipped renameme.txt.

Here’s a sample link to download LINQpad gzipped- http://fwad.herokuapp.com/ZCefeMBka4a7lTaY/http://www.linqpad.net/GetFile.aspx?LINQPad

Here’s a sample link to some weird googlecode uncompressed- http://fwad.herokuapp.com/YaTl7a4akBMefeCZ/https://lightpack.googlecode.com/files/PrismatikSetup_5.9.4

Here’s a link to the troublemaker exe that started all this - http://fwad.herokuapp.com/YaTl7a4akBMefeCZ/http://inedo.com/files/buildmaster/sql/3.5.8

Pretty simple, you just call the service with the appropriate route and some path to an exe without the exe on the end and it will send you a file.

Moral of the story: know what you are doing before you start doing it. don’t expect magic tutorials to save you. =)

A few days ago I was reading one of my usual news feeds The Daily WTF and saw Alex’s post (A (Long Overdue) BuildMaster Introduction) on their BuildMaster product. It looked pretty cool so I downloaded it myself and forwarded it to a colleague at another place of employment. Unfortunately for that individual their firewall policy prevents downloading .EXE files so they could not download any of the installs as they were all .EXE files.

So firstly, I would recommend that anyone that wants to have their cool/nifty product downloaded by someone in a large corp should at least *offer* the option of getting a compressed version that is not an MSI or an EXE file. I’ve seen these blocked at a number of institutions and while they are blocked for good reason, if you want to start a guerilla campaign with developers, it’s easier if they can get your stuff.

Secondly, to circumvent this PITA, I present ‘DEX’ which stands for ‘download EXE’. =) If you are one of the frustrated folks who can’t download EXE files, feel free to use this ‘new’ and ‘innovative’ service =P. You will probably want to let me know as I may change the key from time to time.

It’s very simple.

• Here is the URL format - http://royashbrook.apphb.com/dex.ashx?k=[key]&u=[urltoexe]
• It just processes the url string looking for the values of k,u, and one not shown here gzs.
• k must be YaTl7a4akBMefeCZ currently. It’s just a random key now and will change from time to time. I copied it from the text of a password generated here. This is where I remind you again to just drop me a line if you are using it.
• u should be the url to the exe file you want WITHOUT THE .EXE AT THE END.
• The service will then call ‘u’ with an added .EXE at the end and stream it down to you as renameme.txt.
• If you include gzs=1 it will gzip the stream and try and friendly name it for you although that doesn’t work all the time.

Here are some sample URLs:

• BuildMaster - http://royashbrook.apphb.com/dex.ashx?k=YaTl7a4akBMefeCZ&u=http://inedo.com/files/buildmaster/sql/3.5.8
• Some Project on Google Code - http://royashbrook.apphb.com/dex.ashx?k=YaTl7a4akBMefeCZ&u=https://pirate.googlecode.com/files/Pirate-1008
• Old Version of LINQPad - http://royashbrook.apphb.com/dex.ashx?k=YaTl7a4akBMefeCZ&u=http://www.linqpad.net/GetFile.aspx?LINQPad

Codeplex project is at http://dex.codeplex.com/.

so about… oh… 5 years ago, I posted this blog entry:

http://www.drowningintechnicaldebt.com/RoyAshbrook/archive/2007/03/09/passing-csv-to-sql-is-bad-mmkay.aspx

I recently needed to do something where I was passing in something like a csv to a sql proc and was reminded of it, so I looked it up. I noticed that I really didn’t have a good code sample on the old blog. I happened to notice a similar question on stackoverflow from 2010:

http://stackoverflow.com/questions/2312581/comma-delimited-sql-string-need-to-separated/15668344#15668344

So, I thought now would be a good time to post an updated code sample of this. Below is a method of simply passing a string to check (bad) and a way to pass that same value in xml (good). There are lots of ways to split the strings and do other things, but I like this way. Of course you can do this in LINQ now with a .contains pretty easily, but this is a good way to do it in pure sql.

--table with some test data
select * into #mytable from (
    select 1 [id],'one' [value] union all
    select 2,'two' union all
    select 3,'three' union all
    select 4,'four' union all
    select 5,'five') st

--bad way passing raw csv string data in
declare @csv varchar(50)
set @csv = '1,3,5'
declare @sql nvarchar(2000)
set @sql = 'select * from #mytable where id in (' + @csv + ')'
exec sp_executesql @sql

--vs

--xml document with root, using attributes
declare @xml as xml
set @xml = '<root><v id="1" /><v id="3" /><v id="5" /></root>'
select
    t.*
from
    #mytable t join @xml.nodes('//v') x(n)
    on n.value('@id','int') = t.id

--simple collection of xlm nodes
declare @xml2 as xml
set @xml2 = '<v>1</v><v>3</v><v>5</v>'
select
    t.*
from
    #mytable t join @xml2.nodes('/v') x(n)
    on n.value('.','int') = t.id

drop table #mytable