homepage

This issue tracker has been migrated to GitHub , and is currently read-only.
For more information, see the GitHub FAQs in the Python's Developer Guide.

classification
Title: open 'PhysicalDriveN' on windows fails (since python 3.5) with OSError: [WinError 1] Incorrect function
Type: behavior Stage: resolved
Components: IO, Library (Lib), Windows Versions: Python 3.5
process
Status: closed Resolution: duplicate
Dependencies: Superseder: tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
View: 25717
Assigned To: Nosy List: benjamin.peterson, eryksun, paul.moore, pitrou, rokozh, steve.dower, stutzbach, tim.golden, vstinner, zach.ware
Priority: normal Keywords: 3.5regression

Created on 2015年11月16日 21:42 by rokozh, last changed 2022年04月11日 14:58 by admin. This issue is now closed.

Messages (5)
msg254756 - (view) Author: Roman Kozhemiakin (rokozh) Date: 2015年11月16日 21:42
open('\\\\.\\PHYSICALDRIVE1','rb',0)
fails since python 3.5
At the end of _io_FileIO___init___impl function 
_Py_fstat call raise OSError: [WinError 1] Incorrect function
_Py_fstat result not used on windows in this place. 
 440 self->blksize = DEFAULT_BUFFER_SIZE;
---> 441 if (_Py_fstat(self->fd, &fdfstat) < 0)
 442 goto error;
 443 #if defined(S_ISDIR) && defined(EISDIR)
 444 /* On Unix, open will succeed for directories.
 445 In Python, there should be no file objects referring to
 446 directories, so we need a check. */
 447 if (S_ISDIR(fdfstat.st_mode)) {
 448 errno = EISDIR;
 449 PyErr_SetFromErrnoWithFilenameObject(PyExc_IOError, nameobj);
 450 goto error;
 451 }
 452 #endif /* defined(S_ISDIR) */
 453 #ifdef HAVE_STRUCT_STAT_ST_BLKSIZE
 454 if (fdfstat.st_blksize > 1)
 455 self->blksize = fdfstat.st_blksize;
 456 #endif /* HAVE_STRUCT_STAT_ST_BLKSIZE */
msg254784 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2015年11月17日 06:39
As a workaround you can open a file descriptor via os.open:
 >>> import os
 >>> fd = os.open(r'\\.\PhysicalDrive0', os.O_RDONLY | os.O_BINARY)
 >>> os.read(fd, 512)[:8]
 b'3\xc0\x8e\xd0\xbc\x00|\x8e'
> _Py_fstat result not used on windows in this place. 
While it's pretty much useless, it is possible to open a CRT file descriptor for a directory handle:
 >>> h = _winapi.CreateFile('C:\\', 0x80000000, 3, 0, 3, 0x02000000, 0)
 >>> fd = msvcrt.open_osfhandle(h, os.O_RDONLY)
 >>> open(fd)
 Traceback (most recent call last):
 File "<stdin>", line 1, in <module>
 IsADirectoryError: [Errno 21] Is a directory: 3
So calling _Py_fstat shouldn't be skipped on Windows.
The issue is that a raw disk device doesn't support querying information (IRP_MJ_QUERY_INFORMATION), as shown in this local kernel debugging session:
 lkd> !object \GLOBAL??\PhysicalDrive0
 [...]
 Target String is '\Device\Harddisk0\DR0'
 lkd> !object \Device\Harddisk0\DR0
 Object: ffffe001c4fa4500 Type: (ffffe001c3d7bd30) Device
 [...]
 lkd> !devobj ffffe001c4fa4500
 Device object (ffffe001c4fa4500) is for:
 DR0 \Driver\disk DriverObject ffffe001c4fa3310
 [...]
 lkd> !drvobj \Driver\disk 2
 [...]
 Dispatch routines:
 [...]
 [05] IRP_MJ_QUERY_INFORMATION fffff8037251a06c nt!IopInvalidDeviceRequest
 [06] IRP_MJ_SET_INFORMATION fffff8037251a06c nt!IopInvalidDeviceRequest
 [...]
Specifically WinAPI GetFileInformationByHandle calls the system function NtQueryInformationFile. The system call makes an IRP_MJ_QUERY_INFORMATION I/O request of the driver, which for a raw disk device is handled by IopInvalidDeviceRequest. This fails the request with STATUS_INVALID_DEVICE_REQUEST (0xC0000010), which translates to WinAPI ERROR_INVALID_FUNCTION (0x0001). 
Possibly in the case of ERROR_INVALID_FUNCTION, FileIO can just call PyErr_Clear and skip the code that uses fdfstat.
msg254790 - (view) Author: STINNER Victor (vstinner) * (Python committer) Date: 2015年11月17日 10:47
I don't know the physical disk type on Windows. Can you read and write from such "file" type? If no, I don't think that it makes sense to support it in io.FileIO. What do you think?
It looks like the file must be opened with specific options, otherwise it doesn't work. See this question on CreateFile():
https://social.msdn.microsoft.com/Forums/windowsdesktop/en-US/fa2175fe-02f4-4adb-9cb9-5df3d1122cf6/readfile-works-when-handle-opened-with-drive-letterphysical-drive-symlink-but-not-with-device-path
io.FileIO() calls _Py_fstat() for two reasons:
1) raise an error if the path/file descriptor is a directory
2) get the block size
_Py_fstat() is implemented with two Windows calls: GetFileType() & GetFileInformationByHandle().
To solve this issue, we need to have a function telling that the path/file descriptor is a physical disk. In this case, we can skip both checks.
Documentation on Physical Drivers in CreateFile():
https://msdn.microsoft.com/en-us/library/aa363858%28VS.85%29.aspx#physical_disks_and_volumes 
msg254797 - (view) Author: Roman Kozhemiakin (rokozh) Date: 2015年11月17日 11:55
>I don't know the physical disk type on Windows. Can you read and write from such "file" type?
Yes this "files" can be readed and writed (with restriction - size of the data must be a multiple of the sector size)
in python 3.4.3 open,read,write,seek - works fine (with raw usb disk)
for python 3.5 as workaround I replaced these functions to os.xxxx
>2) get the block size
st_blksize is not implemented under windows
msg260065 - (view) Author: Eryk Sun (eryksun) * (Python triager) Date: 2016年02月11日 01:43
I'm closing this as a duplicate of issue 25717. Martin's patch fixes this problem since it's just another case of _Py_fstat failing for a valid file. In this case the problem is volume and disk files (e.g. r"\\.\C:" and r"\\.\PhysicalDrive1") that don't support querying file information.
History
Date User Action Args
2022年04月11日 14:58:23adminsetgithub: 69825
2016年02月11日 01:43:11eryksunsetstatus: open -> closed
superseder: tempfile.TemporaryFile fails when dir option set to directory residing on host OS mount
messages: + msg260065

resolution: duplicate
stage: test needed -> resolved
2015年11月17日 11:55:08rokozhsetmessages: + msg254797
2015年11月17日 10:47:45vstinnersetmessages: + msg254790
2015年11月17日 06:39:23eryksunsetnosy: + eryksun
messages: + msg254784
2015年11月16日 22:09:20serhiy.storchakasetnosy: + vstinner
2015年11月16日 21:45:11SilentGhostsetkeywords: + 3.5regression
nosy: + stutzbach, pitrou, paul.moore, tim.golden, benjamin.peterson, zach.ware, steve.dower

components: + Windows, IO
stage: test needed
2015年11月16日 21:42:09rokozhcreate

AltStyle によって変換されたページ (->オリジナル) /