Paperclip expiring_url and "Request has expired"

2015-04-28T00:00:00Z

I hit a weird error today. My project stores user uploads on S3 with private permissions, and we use the Paperclip expiring_url method to let people download them. This started failing with a error like this:

<Error>
  <Code>AccessDenied</Code>
  <Message>Request has expired</Message>
  <Expires>2015-04-28T18:30:51Z</Expires>
  <ServerTime>2015-04-28T18:32:52Z</ServerTime>
  <RequestId>BLAHBLAHBLAHBLAH</RequestId>
  <HostId>
    MOREBLAHMOREBLAHMOREBLAHMOREBLAHMOREBLAHMOREBLAHMOREBLAH
  </HostId>
</Error>

The problem was that this error occurred immediately, even though we were generating the URLs with a 10 minute expiration time. Well, it turns out S3 was living in the future by 7 minutes (according to ServerTime above), and our own app was living in the past by another 5. So 5 + 7 = 12 and our URLs were expired before we even generated them. 10 minutes is the example time in the Paperclip documentation linked above, but I guess it’s too short. Watch out for clock skew out there folks!

Paperclip with Server-Side Files

2013-06-07T00:00:00Z

Several times in the last few years I’ve built Rails sites that needed to store files via Paperclip that were not uploaded by a user, but generated programmatically on the server side, for instance PDF or Excel reports built from something in the database. There’s not much documentation on this, and my first effort led to this StackOverflow question. The first solution is not perfect, but it sometimes works:

user.photo = photo_bytes
user.photo.instance_write(:file_name, photo_file_name)
user.save!

But sometimes that won’t do, because Paperclip uses the file name to determine what kind of file you’ve got. For instance, suppose you’re generating a spreadsheet with the WriteExcel gem:

io = StringIO.new
xls = WriteExcel.new(io)
# ...
xls.close
report.spreadsheet = io.string

This will raise a NoHandlerError in the last line. The solution is to make sure you give Paperclip something with a filename, not just the raw binary string.

The easy way out would be to write your spreadsheet to a temp file, then point Paperclip at it. But writing and reading a temp file adds risk of failure, increases disk IO, and slows things down. I’m stubborn, so I really wanted to keep everything in memory.

My solution is based on the second answer to that StackOverflow question above. If we give Paperclip an IO object with a method called original_filename, it will do the right thing. So let’s define this class:

class NamedStringIO < StringIO

  def initialize(data, filename, content_type=nil)
    super(data)
    @filename = filename
    @content_type = content_type
  end

  def original_filename
    @filename
  end

  def content_type
    @content_type
  end

end

Now we can change our Paperclip code to this:

io = StringIO.new
xls = WriteExcel.new(io)
# ...
xls.close
report.spreadsheet = NamedStringIO.new(
  io.string,
  xls_filename,
  'application/vnd.ms-excel')

That code makes Paperclip happy, and it lets us generate a named “file” that never hits the disk.

Note I’m also adding a content_type method. Paperclip will accept an instance without that, but if you are storing the files on S3, it will save the file with a generic content type like application/octet-stream, and that can cause problems when serving the file to web browsers. If you provide a content type here, it will get stored on S3 correctly.

It would be even better to create our NamedStringIO at the top, so we don’t have that extra StringIO we give to WriteExcel. For some reason I couldn’t get WriteExcel to accept my NamedStringIO, so I had to copy things around a bit. Oh well, good enough for me!

Illuminated Computing

Paperclip expiring_url and "Request has expired"

Paperclip with Server-Side Files