How to upload a json file with secret keys to Heroku

Question

I'm building a rails app that pulls data from Google Analytics using the Google Api Client Library for Ruby.

I'm using OAuth2 and can get everything working in development on a local machine. My issue is that the library uses a downloaded file, client_secrets.json, to store two secret keys.

Problem:I'm using Heroku and need a way to get the file to their production servers.

I don't want to add this file to my github repo as the project is public.

If there is a way to temporarily add the file to git, push to Heroku, and remove from git that would be fine. My sense is that the keys will be in the commits and very hard to prevent from showing on github.

Tried: As far I can tell you cannot SCP a file to Heroku via a Bash console. I believe when doing this you get a new Dyno and anything you add would be only be temporary. I tried this but couldn't get SCP to work properly, so not 100% sure about this.

Tried: I looked at storing the JSON file in an Environment or Config Var, but couldn't get it to work. This seems like the best way to go if anyone has a thought. I often run into trouble when ruby converts JSON into a string or hash, so possibly I just need guidance here.

Tried: Additionally I've tried to figure out a way to just pull out the keys from the JSON file, put them into Config Vars, and add the JSON file to git. I can't figure out a way to put ENV["KEY"] in a JSON file though.

---

Example Code The Google library has a method that loads the JSON file to create an authorization client. The client then fetches a token (or gives a authorization url).

client_secrets = Google::APIClient::ClientSecrets.load('client_secrets.json')
auth_client = client_secrets.to_authorization

** note that the example on google page doesn't show a filename because it uses a default ENV Var thats been set to a path

I figure this would all be a lot easier if the ClientSecrets.load() method would just take JSON, a string or a hash which could go into a Config Var.

Unfortunately it always seems to want a file path. When I feed it JSON, a string or hash, it blows up. I've seen someone get around the issue with a p12 key here, but I'm not sure how to replicate that in my situation.

Haven't Tried: My only other though (aside from moving to AWS) is to put the JSON file on AWS and have rails pull it when needed. I'm not sure if this can be done on the fly or if the file would need to be pulled down when the rails server boots up. Seems like too much work, but at this point I've spend a few hours on it so ready to attempt.

This is the specific controller I am working on: https://github.com/dladowitz/slapafy/blob/master/app/controllers/welcome_controller.rb

Answer 1

By search github I found that someone had used a different method that used a JSON string as an argument rather than a file path: Google::APIClient::ClientSecrets.new(JSON.parse(ENV['GOOGLE_CLIENT_SECRETS']))

This lets me wrap up the JSON into an ENV VAR. The world makes sense again.

• paullj1: How are you storing your json data in the env var (i.e., newlines, quotes)? I can import it with this line in the console, but I tried setting it up exactly as you have it in your controller and I'm getting a "no implicit conversion of Symbol into Integer" inside the initialize function for ClientSecrets when calling this line. I have no problems trying to access the item it's failing on so it must be getting lost somewhere in between.
• David Ladowitz: I'm using the figaro gem and then storing my environmental variables in application.yml. In this file I've set up the variable like so: GOOGLE_CLIENT_SECRETS: '{"web":{"client_id":"XXXXXX-XXXXXXXXXXXX.apps.googleusercontent.com","project_id":"XXXX-XXXX","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://accounts.google.com/o/oauth2/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"XXXXXXXXXX","redirect_uris":["http://localhost:3000/oauthredirect"]}}'
• David Ladowitz: @paullj1 In Heroku the Config Var value is almost the same, but I removed the most outer single quotes. One thing to notice in my solution above it that I found a different method that does't take a file name. Originally I was using Google::APIClient::ClientSecrets.load() Now I am using Google::APIClient::ClientSecrets.new()

Answer 2

As discussed in this thread, rather than supplying a path to a json key file you can set three ENV variables instead:

GOOGLE_ACCOUNT_TYPE=service_account
GOOGLE_PRIVATE_KEY=XXX
GOOGLE_CLIENT_EMAIL=XXX

Source here.

• Nick: This seemed to work using the Cloud Natural Language API and saved me a lot of messing around.
• gezquinndesign: Hi @Nick - sorry to dredge up old work - I'm in the same predicament and wondering how you resolved this issue for the Natural Language API?
• Nick: @gezquinndesign This was a non-production app, but in the end I ended up just setting it ENV['GOOGLE_PRIVATE_KEY'] etc in development.rb and it worked...

Answer 3

I ran into this same problem using Google API. I ended up using openssl to assign a new very secret passphrase to the p12 file, storing that new file in the repo, and then putting the passphrase into app secrets and on Heroku env variables.

This way, the file is in the repo but it can't be accessed/read without the passphrase.

This post was helpful in changing the default google p12 passphrase from 'notasecret' to something secure.

def authorize!
 @client.authorization = Signet::OAuth2::Client.new(
 #...
 :signing_key => key
 )
end
def key
 Google::APIClient::KeyUtils.load_from_pkcs12(key_path, ENV.fetch('P12_PASSPHRASE'))
end
def key_path
 "#{Rails.root}/config/google_key.p12"
end

Answer 4

Using Rails 7, I encrypted the JSON credentials like so

I first ran bin/rails credentials:edit -e development

Then added my credentials:

 omniauth:
 google_oauth2:
 client_secrets: {"web":{"client_id":"my-client-id","project_id":"my-project-id","auth_uri":"https://accounts.google.com/o/oauth2/auth","token_uri":"https://oauth2.googleapis.com/token","auth_provider_x509_cert_url":"https://www.googleapis.com/oauth2/v1/certs","client_secret":"my-client-secret","redirect_uris":["http://localhost:3000/users/auth/google_oauth2/callback","http://localhost:3000/contacts/gmail/callback"]}}

The I used it with ClientSecrets like so:

 def client_secrets
 @client_secrets ||= Google::APIClient::ClientSecrets.new(client_secrets_json)
 end
 def client_secrets_json
 Rails.application.credentials.dig(:omniauth, :google_oauth2, :client_secrets)
 end

Answer 5

For anyone stumbling upon this in the future, this is how I fixed it:

 # GOOGLE_PLAY_CLIENT_SECRETS_JSON = "{\"web\":{\"client_id\":\"XXXXX", ...}}"
 json = ENV['GOOGLE_PLAY_CLIENT_SECRETS_JSON']
 file = File.open('client_secrets.json', 'w') do |f|
 f.write(json)
 end
 CLIENT_SECRETS = Google::APIClient::ClientSecrets.load('client_secrets.json')