GPG upload of newly-changed key fails because we cache the old key

Bug Description

The upload of a newly-changed GPG key fails and keeps failing for a long time after we update the key. James H. has provided us with a good hypothesis:

<quote>
The GPGHandler.retrieveKey() code does the following:

1. try to get key from local keyring

2. if that fails, grab it from our keyserver

So if someone tries to add a sign-only key we'll cache that key in the app server's $GNUPGHOME. If the user adds an encryption key, waits for the keyserver to synchronise and tries to add it again, LP will still get the old cached version unless one of the following holds:

* they hit a different app server
* the app server process has been restarted (which creates a new $GNUPGHOME)

These two variables could indicate why the problem appears to be intermittent.
</unquote>

The solution may involve writing code to invalidate the cache, and always invalidating when the end-user is uploading a key -- it is still useful for other situations, of course.

Assigning James H for the time being.

The same problem is occuring with me..

I will reproduce what i did, if that helps you

1) I had previously registered with Key ID : 995D6B13
and email address <email address hidden> [worked]

2) i then changed my email id to <email address hidden> [worked]

3) I wanted to change my PGP key as i had lost my earlier one
i tried two
a) fingerprint: 191B CFBF 5C87 B0A5 BA7B 2A1A 8BE4 F7F7 7019 9A55
ID: 70199A55
b) 3E47 9345 7436 E11E B6EC 6D15 A84E E67E 9D53 6D7B
ID: 9D536D7B

The first one has <email address hidden> as email and the second on has <email address hidden>

screenshot : http://paste.ubuntu-nl.org/8731

Hi James,

Does your gpg backend address those cache issues? We had an user who was unable to revalidate his key and I thought it could be caused by the caching.
You can see here:
https://launchpad.net/products/launchpad/+ticket/1020
the steps we took to try to import his key into launchpad. So far we had no sucess.

I don't think this is fixed yet. The pygpgme integration work I did didn't change this aspect of gpghandler.py.

This is still an issue. https://answers.edge.launchpad.net/launchpad/+question/50469

This will consider fixing this as a part of a larger effort to improve Launchpad's integration of pygpgme

Note that this is a side-effect of having a permanent GNUPGHOME in appservers, this problem doesn't happen with the cronjobs (which creates a new GNUPGHOME every run).

My impression is that we could simply modify IGPGHandler.retrieveKey() to accept a 'force' argument which would force it to import the key from the keyserver even if it is already present in the local keyring, and obviously use that for sensible operations like validating keys.

We may find out that we need to re-import keys in many places in order to always use up-to-date keys. In this case we have to investigate the impacts of the extra requests on the internal keyserver (which is single-threaded).

Still an issue, three years since the last comment. There's a bunch of documentation talking about how to upload an updated key after changing expiration -- and it will show up on the keyserver -- but there's no way to update the cached key.

Since gpg is perfectly happy importing a key that already exists, and will update the key in-place when doing so, how about simply removing the sanity check triggering the "key exists" warning, and simply import a fingerprint _always_? This wouldn't require extra switches at all.

Today I had the same problem. I extended the key lifetime, but I can't reactivate the key as the server always complains that the key is expired. Additionally the key has a revoked email address due to discontinuing service by Berlios.

Same as the comment#9. I renew my public key but Launchpad always complains my key has been expired. My key fingerprint 2BB2 0A20 01E3 26F8 37E8 2FEF D371 FA62 B582 92C2

https://code.launchpad.net/~cjwatson/launchpad/no-auto-key-retrieve/+merge/364100 doesn't quite fix this, but I think it will deal with situations related to uploading to PPAs using a renewed key, and it should make it easier to fix the rest of it.

How often do the app-server(s) restart ? Is it only a few hours/days and thus acceptable to just wait for ?

(same issue with F2F0E746E8CF632E9A7E5EAFFEE8E662276132D0 which I renew yearly)

As Todd said, gpg can merge 2 key versions just fine, so allowing updates (even with a throttle if necessary) imo would be a good thing.

At present they're restarted any time we do a code deployment, which is usually once or twice a week at the moment.

I still cannot link the key. Does it take it from keyserver.ubuntu.com ? I noticed that the mainkey [SC] 0x276132D0 there in fact does not include the latest signature that would still be valid.
I published that key to several keyserver, on all others I've checked it is up to date, but not on Ubuntu.

When I try to reupload it (the bundle with the latest sig) on https://keyserver.ubuntu.com/pks/add I get that it was ignored:

{"inserted":null,"updated":null,"ignored":["rsa3072/f2f0e746e8cf632e9a7e5eaffee8e662276132d0"]}

What could be the reason for that / is there a issuetracker for the keyserver site too?