Re: [Python-Dev] subprocess not escaping "^" on Windows

On Sun, Jan 7, 2018 at 6:48 PM, Christian Tismer wrote:> That is true.> list2cmdline escapes partially, but on NT and Windows10, the "^" must> also be escaped, but is not. The "|" pipe symbol must also be escaped> by "^", as many others as well.>> The effect was that passing a rexexp as para

Re: [Python-Dev] subprocess not escaping "^" on Windows

On Mon, Jan 8, 2018 at 9:26 PM, Steve Dower wrote: > On 09Jan2018 0744, eryk sun wrote: >> >> It's common to discourage using `shell=True` because it's considered >> insecure. One of the reasons to use CMD in Windows is that it tries >> ShellExecuteEx i

Re: [Python-Dev] Python environment registration in the Windows Registry

On Wed, Feb 3, 2016 at 10:46 AM, Steve Dower wrote:>> sys.path.extend(read_subkeys(fr'HKCU\Software\Python\PythonCore\{sys.winver}\PythonPath\**'))> sys.path.extend(read_subkeys(fr'HKLM\Software\Python\PythonCore\{sys.winver}\PythonPath\**')) It seems like a bug (in spirit at least) that this

Re: [Python-Dev] Python environment registration in the Windows Registry

On Wed, Feb 3, 2016 at 7:33 PM, Eric Snow wrote:> Just wanted to quickly point out another use of the WIndows registry> in Python: WindowsRegistryFinder [1]. This is an import "meta-path"> finder that locates modules declared (*not* defined) in the registry.> I'm not familiar with the Windows

Re: [Python-Dev] When does `PyType_Type.tp_alloc get assigned to PyType_GenericAlloc ?

On Sun, Feb 7, 2016 at 7:58 AM, Randy Eels wrote:>> Yet, I can't seem to understand where and when does the `tp_alloc` slot of> PyType_Type get re-assigned to PyType_GenericAlloc. Does that even happen?> Or am I missing something bigger? _Py_InitializeEx_Private in Python/pylifecycle.c calls

Re: [Python-Dev] Windows: Remove support of bytes filenames in the os module?

On Mon, Feb 8, 2016 at 2:41 PM, Chris Barker wrote:> Just to clarify -- what does it currently do for bytes? IIUC, Windows uses> UTF-16, so can you pass in UTF-16 bytes? Or when using bytes is is assuming> some Windows ANSI-compatible encoding? (and what does it return?) UTF-16 is used in the

Re: [Python-Dev] Windows: Remove support of bytes filenames in the os module?

On Tue, Feb 9, 2016 at 3:21 AM, Victor Stinner wrote: > 2016年02月09日 1:37 GMT+01:00 eryk sun : >> For example, in codepage 932 (Japanese), it's an error if a lead byte >> (i.e. 0x81-0x9F, 0xE0-0xFC) is followed by a trailing byte with a >> value less than 0x40 (note that

Re: [Python-Dev] Windows: Remove support of bytes filenames in the os module?

On Tue, Feb 9, 2016 at 3:22 AM, Victor Stinner wrote: > 2016年02月09日 1:37 GMT+01:00 eryk sun : >> For example, in codepage 932 (Japanese), it's an error if a lead byte >> (i.e. 0x81-0x9F, 0xE0-0xFC) is followed by a trailing byte with a >> value less than 0x40 (note that

Re: [Python-Dev] Windows: Remove support of bytes filenames in theos module?

On Wed, Feb 10, 2016 at 2:30 PM, Andrew Barnert via Python-Dev wrote:> [^3]: Say you write a program that assumes it will only be run on Shift-JIS > systems, and you use> CreateFileA to create a file named "ハローワールド". The actual bytes you're sending > are cp436> for "ânâìü[âÅü[âïâh", so the

Re: [Python-Dev] [Windows] how to prevent the wrong version of zlib1.dll to be used by lib-dynload modules

On Mon, Jul 23, 2018 at 2:31 PM, Eric Le Lay wrote:>> I encountered a problem with the Windows packaging of gPodder[1]> using msys2: Are you using regular Windows Python with msys2, or their custom port? I installed msys2 and used pacman to install Python 3.6. The msys2 environment names libr

Re: [Python-Dev] Tests failing on Windows with TESTFN

On Sat, Jul 28, 2018 at 5:20 PM, Tim Golden wrote:>> I've got a mixture of Permission (winerror 13) & Access errors (winerror 5) EACCES (13) is a CRT errno value. Python raises PermissionError for EACCES and EPERM (1, not used). It also does the reverse mapping for WinAPI calls, so PermissionEr

Re: [Python-Dev] Tests failing on Windows with TESTFN

On Sat, Jul 28, 2018 at 9:17 PM, Jeremy Kloth wrote:>> *PLEASE*, don't use tempfile to create files/directories in tests. It> is unfriendly to (Windows) buildbots. The current approach of> directory-per-process ensures no test turds are left behind, whereas> the tempfile solution slowly fil

Re: [Python-Dev] Tests failing on Windows with TESTFN

On Sun, Jul 29, 2018 at 9:13 AM, Tim Golden wrote:>> For an example:>> http://tjg.org.uk/test.log>> Thinkpad T420, 4Gb, i5, SSD>> Recently rebuilt and reinstalled: Win10, VS2017, TortoiseGit, standard> Windows Antimalware, usual developer tools. That particular run was done> with the lap

Re: [Python-Dev] Tests failing on Windows with TESTFN

On Sun, Jul 29, 2018 at 12:35 PM, Steve Dower wrote:>> One additional thing that may help (if support.unlink doesn't already do it)> is to rename the file before deleting it. Renames are always possible even> with open handles, and then you can create a new file at the original name. Renaming

Re: [Python-Dev] Tests failing on Windows with TESTFN

On Sun, Jul 29, 2018 at 2:21 PM, Jeremy Kloth wrote:>> try:> os.rename(new_file.name, self._path)> except FileExistsError:> -os.remove(self._path)> +temp_name = _create_temporary_name(self._path)> +os.rename(self._path, temp

Re: [Python-Dev] Questions about signal handling.

On Fri, Sep 21, 2018 at 6:10 PM, Victor Stinner wrote:>> Moreover, you can get the signal while you don't hold the GIL :-) Note that, in Windows, SIGINT and SIGBREAK are implemented in the C runtime and linked to the corresponding console control events in a console application, such as python.

Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?

On 1/17/19, Steven D'Aprano wrote:>> I understand that the only way to pass the address of an object to> ctypes is to use that id. Is that intentional? It's kind of dangerous to pass an object to C without an increment of its reference count. The proper way is to use a simple pointer of type "

Re: [Python-Dev] ctypes: is it intentional that id() is the only way to get the address of an object?

On 1/18/19, Steven D'Aprano wrote: > On Thu, Jan 17, 2019 at 07:50:51AM -0600, eryk sun wrote: >> >> It's kind of dangerous to pass an object to C without an increment of >> its reference count. > > "Kind of dangerous?" How dangerous? I take that

Re: [Python-Dev] Adding test.support.safe_rmpath()

On 2/16/19, Richard Levasseur wrote:>> First: The tempfile module is a poor fit for testing (don't get me wrong,> it works, but its not *nice for use in tests*)*.* This is because:> 1. Using it as a context manager is distracting. The indentation signifies> a conceptual scope the reader needs

Re: [Python-Dev] Remove tempfile.mktemp()

On 3/19/19, Victor Stinner wrote:>> When I write tests, I don't really care of security, but> NamedTemporaryFile caused me many troubles on Windows: you cannot> delete a file if it's still open in a another program. It's way more> convenient to use tempfile.mktemp(). Opening the file again f

Re: [Python-Dev] Remove tempfile.mktemp()

On 3/20/19, Anders Munch wrote:>> You are right, I must have mentally reversed the polarity of the delete> argument. And I didn't realise that the access right on a file had the> power to prevent itself from being removed from the folder that it's in. I> thought the access flags were a prop

Re: [Python-Dev] Remove tempfile.mktemp()

On 3/20/19, Greg Ewing wrote:> Antoine Pitrou wrote:>>> How is it more secure than using mktemp()?>> It's not, but it solves the problem someone suggested of another> program not being able to access and/or delete the file. NamedTemporaryFile(delete=False) is more secure than naive use of m

Re: [Python-Dev] Remove tempfile.mktemp()

On 3/23/19, Cameron Simpson wrote:>> Also, the common examples are attackers who are not the user making the> tempfile, in which case the _default_ mktemp is sort of secure with the> above because it gets made in /tmp which on a modern POSIX system> prevents _other_ uses from removing/renamin

[Python-Dev] Re: What to do about invalid escape sequences

On 8/5/19, Steve Dower wrote:>> though I do also see many people bitten by FileNotFoundError> because of a '\n' in their filename. Thankfully the common filesystems used in Windows reserve ASCII control characters in filenames (except not in stream names or named-pipe names). So a mistaken str

[Python-Dev] Re: What to do about invalid escape sequences

On 8/7/19, Steve Dower wrote:>> * change the PyErr_SetExcFromWindowsErrWithFilenameObjects function to> append (or chain) an extra message when either of the filenames contains c> control characters (or change OSError to do it, or the default> sys.excepthook) On a related note for Windows, i

[Python-Dev] Re: os.scandir bug in Windows?

On 10/15/20, Rob Cliffe via Python-Dev wrote:>> TLDR: In os.scandir directory entries, atime is always a copy of mtime> rather than the actual access time. There are inconsistencies in various scenarios between between the stat info from the directory entry and the stat info from the File Cont

[Python-Dev] Re: os.scandir bug in Windows?

On 10/19/20, Steve Dower wrote:> On 15Oct2020 2239, Rob Cliffe via Python-Dev wrote:>> TLDR: In os.scandir directory entries, atime is always a copy of mtime>> rather than the actual access time.>> Correction - os.stat() updates the access time to _now_, while> os.scandir() returns the last

[Python-Dev] Re: os.scandir bug in Windows?

On 10/19/20, Steve Dower wrote:> On 19Oct2020 1242, Steve Dower wrote:>> On 15Oct2020 2239, Rob Cliffe via Python-Dev wrote:>>> TLDR: In os.scandir directory entries, atime is always a copy of mtime>>> rather than the actual access time.>>>> Correction - os.stat() updates the access time to

[Python-Dev] Re: os.scandir bug in Windows?

On 10/19/20, Steve Dower wrote:>> Resolving the path is the most expensive part, even if the file is not> opened (I've been working with the NTFS team on this area, and we've> been benchmarking/analysing all of it). If you say it's been extensively benchmarked and there's no direct way around

[Python-Dev] Re: os.scandir bug in Windows?

On 10/19/20, Greg Ewing wrote:> On 20/10/20 4:52 am, Gregory P. Smith wrote:>> Those of us with a traditional posix filesystem background may raise>> eyeballs at this duplication, seeing a directory as a place that merely>> maps names to inodes>> This is probably a holdover from MS-DOS, wher

[Python-Dev] Re: os.scandir bug in Windows?

On 10/26/20, Victor Stinner wrote:> Le lun. 19 oct. 2020 à 13:50, Steve Dower a écrit> :>> Feel free to file a bug, but we'll likely only add a vague note to the>> docs about how Windows works here rather than changing anything.>> I agree that this surprising behavior can be documented. Att

[Python-Dev] Re: os.scandir bug in Windows?

On 10/28/20, Stephen J. Turnbull wrote:>> Note: you can "fix" directory updates by mounting the filesystem r/o. Mounting the filesystem as readonly is the extreme case. Popular Unix systems support a "noatime" mount option that disables updating file access times, unless one of the other timest

[Python-Dev] Re: PEP 597: Add optional EncodingWarning

On 2/11/21, Inada Naoki wrote:>> There is little difference between `encoding=None` and> `encoding=locale.getpreferredencoding(False)`. The difference is:>> * When Python is using Windows, and> * When when the file is console, and> * (for open()) When PYTHONLEGACYWINDOWSSTDIO is set> * (fo

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

On 2/28/21, Oscar Benjamin wrote:>> - It is possible to configure a default version (although I think you> have to do it with an environment variable) The py launcher in Windows supports a "py.ini" file beside the executable and in %LocalAppData%. The equivalent of the PY_PYTHON, PY_PYTHON2, a

[Python-Dev] Re: Have virtual environments led to neglect of the actual environment?

On 2/28/21, Oscar Benjamin wrote:>> Oh, okay. So does that mean that it's always on PATH unless the user> *explicitly unticks* the "install the launcher" box for both single> user and all user installs? If the launcher gets installed, it will be available in PATH. IIRC, the installer only all

Re: [Python-Dev] Status of the Argument Clinic DSL

On Thu, Aug 4, 2016 at 11:33 PM, Alexander Belopolsky wrote:>> On Thu, Aug 4, 2016 at 7:12 PM, Larry Hastings wrote:>>>> C extension functions get the module passed in automatically, but this is>> done internally and from the Python level you can't see it.>> Always something new to learn!

Re: [Python-Dev] File system path encoding on Windows

On Mon, Aug 22, 2016 at 3:58 PM, Steve Dower wrote:> All MSVC users have been pushed towards Unicode for many years. The .NET> Framework has defaulted to UTF-8 its entire existence. The use of code pages> has been discouraged for decades. We're not going first :) I just wrote a simple function

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

I have some suggestions. With ReadConsoleW, CPython can use the pInputControl parameter to set a CtrlWakeup mask. This enables a Unix-style Ctrl+D for ending a read without having to press enter. For example:>>> CTRL_MASK = 1 << 4>>> inctrl = (ctypes.c_ulong * 4)(16, 0, CTRL_MASK, 0)

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

On Mon, Sep 5, 2016 at 7:54 PM, Steve Dower wrote: > On 05Sep2016 1234, eryk sun wrote: >> >> Also, the console is UCS-2, which can't be transcoded between UTF-16 >> and UTF-8. Supporting UCS-2 in the console would integrate nicely with >> the filesystem PEP. It m

Re: [Python-Dev] PEP 528: Change Windows console encoding to UTF-8

On Mon, Sep 5, 2016 at 9:45 PM, Steve Dower wrote:>> So it works, though the behaviour is a little strange when you do it from> the interactive prompt:> sys.stdin.buffer.raw.read(1)> ɒprint('hi')> b'\xc9' hi sys.stdin.buffer.raw.read(1)> b'\x92'>> What happens here is

Re: [Python-Dev] ctypes, memory mapped files and context manager

On Thu, Jan 5, 2017 at 2:37 AM, Nick Coghlan wrote:> On 5 January 2017 at 10:28, Hans-Peter Jansen wrote:>> In order to get this working properly, the ctypes mapping needs a method to>> free the mapping actively. E.g.:>>>> @contextmanager>> def map_struct(m, n):>> m.resize(n * mmap.PAG

Re: [Python-Dev] ctypes, memory mapped files and context manager

On Thu, Jan 5, 2017 at 11:28 PM, Hans-Peter Jansen wrote:> Leaves the question, how stable this "interface" is?> Accessing _objects here belongs to voodoo programming practices of course, but> the magic is locally limited to just two lines of code, which is acceptable in> order to get this con

Re: [Python-Dev] ctypes, memory mapped files and context manager

On Sun, Jan 8, 2017 at 8:25 AM, Armin Rigo wrote:>> c_raw = ctypes.PYFUNCTYPE(ctypes.c_void_p, ctypes.c_void_p)(lambda p: p) Use ctypes.addressof.> addr = c_raw(ctypes.pointer(T.from_buffer(m)))> b = ctypes.cast(addr, ctypes.POINTER(T)).contents ctypes.cast uses an FFI call. In

[Python-Dev] Re: What is __int__ still useful for?

On 10/14/21, Antoine Pitrou wrote:> On 2021年10月13日 17:00:49 -0700> Guido van Rossum wrote:>>>> so int() can't call __trunc__ (as was explained earlier in>> the thread). I guess this was meant to be "*just* call __trunc__". It's documented that the int constructor calls the initializing

[Python-Dev] Re: What is __int__ still useful for?

On 10/15/21, Mark Dickinson wrote:>> the proposal would be to remove that special role of `__trunc__` and> reduce the `int` constructor to only looking at `__int__` and `__index__`. For Real and Rational numbers, currently the required method to implement is __trunc__(). ISTM that this proposa

[Python-Dev] Re: Preventing Unicode-related gotchas (Was: pre-PEP: Unicode Security Considerations for Python)

On 11/13/21, Terry Reedy wrote:> On 11/13/2021 4:35 PM, [email protected] wrote:>>>> _PLACEHOLDER_LEN = 12>>>> def _shorten(s, prefixlen, suffixlen):>>>> skip = len(s) - prefixlen - suffixlen>>>> if skip> _PLACEHOLDER_LEN:>>>> s = '%s[%d chars]%s' % (s[:prefixlen], skip, s[le

[Python-Dev] Re: Improvements to the sys.path initialization documentation

On 3/4/22, Victor Stinner wrote:> it would be nice to move the last bits of the sys.path initialization> from the site module to the getpath module. It's unpleasant to> have a different sys.path depending if the site module is loaded> or not. I don't understand. The site packages directories,

[Python-Dev] Re: Restrict the type of __slots__

On 3/18/22, Ronald Oussoren via Python-Dev wrote:>> - if __slots__ is a dict keep it as is> - Otherwise use tuple(__slots__) while constructing the class and store that> value in the __slots__ attribute of the class If this is just for pydoc, then it can be updated with new behavior. For exam

[Python-Dev] Re: Restrict the type of __slots__

On 3/19/22, Eryk Sun wrote: > On 3/18/22, Ronald Oussoren via Python-Dev wrote: >> >> - if __slots__ is a dict keep it as is >> - Otherwise use tuple(__slots__) while constructing the class and store >> that >> value in the __slots__ attribute of the class >

[Python-Dev] Re: Add -P command line option to not add sys.path[0]

On 4/26/22, Victor Stinner wrote:>> There are 4 main ways to run Python:>> (1) python -m module [...]> (2) python script.py [...]> (3) python -c code [...]> (4) python [...]>> (1) and (2) insert the directory of the module/script at sys.path[0]. Running a module with -m inserts the curre

[Python-Dev] Re: glob's new include_hidden parameter

On 9/12/22, Mats Wichmann wrote:>> If `include_hidden` is true, the patterns '*', '?', '**' will> match hidden directories. Shouldn't this explain what a "hidden directory" is? For example, a Windows user may think this means a directory with FILE_ATTRIBUTE_HIDDEN set, but that's not what's m

[Python-Dev] Re: Debugging of native extensions on windows

On 3/13/23, Rokas Kupstys wrote:> I eventually stumbled on to process list showing> ".venv/Scripts/python.exe" having spawned a subprocess... Which led me> to "PC/launcher.c" which is what ".venv/Scripts/python.exe" really is. For a standard Python installation, you can create a virtual enviro

[Python-Dev] Re: Small lament...

On 4/1/23, Skip Montanaro wrote:> Just wanted to throw this out there... I lament the loss of waking up on> April 1st to see a creative April Fool's Day joke on one or both of these> lists, often from our FLUFL... Maybe such frivolity still happens, just not> in the Python ecosystem? I though

[Python-Dev] Re: What to do about invalid escape sequences

On 8/9/19, Steven D'Aprano wrote:>> I'm also curious why the string needs to *end* with a backslash. Both of> these are the same path:>> C:\foo\bar\baz\> C:\foo\bar\baz The above two cases are equivalent. But that's not the case for the root directory. Unlike Unix, filesystem namesp

[Python-Dev] Re: What to do about invalid escape sequences

On 8/10/19, eryk sun wrote: > > The per-logon directory is located at "\\Sessions\0円\\DosDevices\\ Session ID>". In the Windows API, it's accessible as "//?/" or "//./", > or with any mix of forward slashes or backslashes, but only the >

[Python-Dev] Re: What to do about invalid escape sequences

On 8/10/19, Rob Cliffe via Python-Dev wrote: > On 10/08/2019 11:50:35, eryk sun wrote: >> On 8/9/19, Steven D'Aprano wrote: >>> I'm also curious why the string needs to *end* with a backslash. Both of >>> these are the same path: >>> >>>

[Python-Dev] Re: os.add_dll_directory and DLL search order

On 6/22/20, Steve Dower wrote:>> What is likely happening here is that _sqlite3.pyd is being imported> before _mapscript, and so there is already a SQLITE3 module in memory.> Like Python, Windows will not attempt to import a second module with the> same name, but will return the original one.