homepage

When a ~/.netrc file has a password with non-word
characters parsing fails. Since it is recommended to
use punctuation characters in passwords, this means
most netrc files can't be parsed. An example of a line
in ~/.netrc that fails:
machine piet.puk.com 	login foo 	password bar!
Additionally, entries in netrc may not have a login
name (e.g., for mail servers). These entries should be
silently skipped. An example of a line that fails:
machine mail password fruit
The included diff is a partial solution. It allows all
ASCII punctuation characters to be used in passwords. 
Non-ASCII characters should probably also be allowed.

Logged In: YES 
user_id=80475
Expanding keyspace is generally a good idea; however, the 
significance of meta-characters is bound to bite someone in 
the behind with a hard to find error. So, please 
reconsider single-quote, double-quote, backslash, 
greaterthan, lessthan, and pipe.
Looking at first part of the patch, consider:
- removing the TODO on line 31
- wrapping the character list in triple-quotes on line 32
- using the r'' form on line 32 to eliminate backslashes in 
the character list
Looking at the second part of the patch, I don't follow (am 
perhaps being daft) why expanding the keyspace necessitates 
changing the login logic.
The idea of allowing non-ASCII characters would be cool if 
the world had already universally accepted Latin-1 coding. 
That conflict is the reason that site.py defaults to ASCII 
encoding instead of handling non-US codings out of the box.

Logged In: YES 
user_id=44345
I think a better solution would be to not use shlex to parse netrc files. 
Netrc files aren't shells. The whitespace is significant if it occurs inside a 
password. I'd just use re.split(r'(\s+)') and restore the password when I 
encounterd the "password" keyword.

Logged In: YES 
user_id=6380
I think Fred knows this code.
I think Eric Raymond (the original author) wrote this as an 
example of his shlex. :-)

Logged In: YES 
user_id=57665
Note that the old Netrc class in the ftplib module has a
different approach at parsing the .netrc file. This might
actually work much better.

Logged In: YES 
user_id=57665
Can someone please do something about this bug? It has been
open for almost a year now and it still can't handle my
netrc file. At least include a temporary fix! My patch
plus the remarks from rhettinger should be sufficient.

Logged In: YES 
user_id=6380
Raymond, can you deal with this or find someone else? (Maybe
the fellow who last patched shlex.py?)

Logged In: YES 
user_id=80475
Revised netrc.py to include the additional ascii punctuation 
characters. Omitted the other logic changes. See 
Lib/netrc.py 1.17.
Since this is more of a feature request than a bug, 
including in Py2.3 but not recommending for backporting.

Logged In: YES 
user_id=6380
Given the size and nature of the patch I have no problem
with a 2.2.3 backport.

Logged In: YES 
user_id=80475
Backported for 2.2.3.
Closing bug.

Logged In: YES 
user_id=44345
This is still not correct, as passwords in .netrc files can't contain spaces.
The netrc module is perhaps a good demonstration of the shlex module,
but I wouldn't rely on it for actual use.

Logged In: YES 
user_id=57665
I am glad the special characters in passwords are now
accepted. But that is only half a fix! My ~/.netrc
contains entries without a "login" field, thus I still
cannot use the netrc module, it bails out at the first line. 
Therefore I have re-opened this issue.
All other programs work just fine with this .netrc file. 
Please at least do not produce the NetrcParseError when the
"login" field is omitted. This can be done by changing the
"else:" above "malformed %s entry" to "elif not password:".
 That is the minimal change to make this module work on my
system.
Note to montanaro: I have not seen a .netrc file that has a
space in the password.

Logged In: YES 
user_id=44345
Passwords with spaces are valid, however I confirmed that the ftp program
which comes with Redhat Linux also gripes about passwords containing 
spaces, so my complaint is probably moot.

Logged In: YES 
user_id=80475
Unassigning, in case someone else wants to explore the 
handling of spaces.

Logged In: YES 
user_id=80475
Instead of skipping lines without login info, write a record 
with login=''.
See netrc.py 1.18.
Will backport to Py2.2.3

I know this is ancient, but the below patch handles spaces in passwords in 2.7.8 and 3.4 for me. If this is worth making into a new bug / proper patch I'm happy to do it.
$ diff -uw /c/Python278/Lib/netrc.py{-orig,}
--- /c/Python278/Lib/netrc.py-orig 2014年10月09日 13:52:36.995286000 +0100
+++ /c/Python278/Lib/netrc.py 2014年10月09日 13:53:05.041286000 +0100
@@ -111,7 +111,23 @@
 "~/.netrc access too permissive: access"
 " permissions must restrict access to only"
 " the owner", file, lexer.lineno)
+ # handle passwords with quoted spaces
+ quote_chars = lexer.quotes
+ removed_chars = []
+ for quote_char in quote_chars:
+ if quote_char in lexer.wordchars:
+ lexer.wordchars = lexer.wordchars.replace(quote_char, '')
+ removed_chars.append(quote_char)
+ try:
+
 password = lexer.get_token()
+
+ for quote_char in quote_chars:
+ if password.startswith(quote_char) and password.endswith(quote_char):
+ password = password[1:-1]
+ finally:
+ for removed_char in removed_chars:
+ lexer.wordchars += removed_char
 else:
 raise NetrcParseError("bad follower token %r" % tt,
 file, lexer.lineno)

Sorry for the whitespace-unaware diff. The attached patch is the real one, with the obvious extra level of indentation around the critical "password = lexer.get_token()" line.

Why is this issue fixed? I still see this problem on 2.7 and 3.4.3.
Can someone please reopen it?
mdengler's patch seems to work fine on my machine on both 2.7 and 3.4.3.

This issue was closed because other FTP programs also did not handle passwords with spaces. If this has subsequently changed (passwords with spaces are now widely accepted by other FTP programs) then the issue could be reopened.

To clarify: other FTP programs handling passwords with spaces *in the .netrc file*.

Anecdotal ( http://stackoverflow.com/a/12675195/2747741 ) evidence suggests other programs do indeed accept spaces, and a cursory browsing of the Perl source code ( http://perl5.git.perl.org/perl.git/blob/HEAD:/cpan/libnet/lib/Net/Netrc.pm#l120 ) indicates Perl at least supports (escaped) spaces in .netrc passwords.
Is there anything that I could do to make the patch I provided acceptable?
If not, could the bug be reopened as 1) the bug description mentions a valid use case that is not handled by the netrc module; and 2) there is precedent for this use case's implementation in other software.

(Hi Bram! :-)
So does your patch also accept escaped spaces? I wonder if one of the problems here may not be that the syntax required to escape special characters isn't specified? That might be acceptable in 2003, not so much in 2016.

Bram's patch for "special" characters is in, mine is the one that allows spaces in .netrc by enabling the parsing of a password field's value that's surrounded by lexer.quotes ( https://hg.python.org/cpython/file/2.7/Lib/shlex.py#l45 ).
This is not the same as Perl's approach (parse the file in a quoted-character-aware way, so quoted spaces don't separate tokens), but was much simpler to implement.
So, in effect, there are no general support for quoting introduced by my patch, only a special case for supporting the entire contents of the password field to be surrounded by the shlex.quote characters.
Would you accept a different (longer, more involved) patch to implement the arbitrary quoting of characters, or an update to this patch to document how the password field is treated and which characters can surround it?

If it is a matter of following "the normal rules" about quoting in a place where we currently don't do that, I think it would be sensible to add it, but IMO it should be the full set of "normal rules". Presumably shlex provides facilities to do that...I haven't looked at the netrc code in quite a while so I don't remember how it all fits together.
As for reopening the issue...there was something that was fixed here, so what we should do instead is open a new issue with your documentation about current reality, a quick summary of this discussion, and a mention of this issue as part of the backstory.

Is there anything blocking this from being really fixed? It's still broken on 3.5.
The patch added two years ago works well for quoted passwords, I think that's good enough, anyway having some support is much better than the current out of the box situation.