2842 – std.file.listdir on OSX produces invalid UTF-8 sequence

D issues are now tracked on GitHub. This Bugzilla instance remains as a read-only archive.
Issue 2842 - std.file.listdir on OSX produces invalid UTF-8 sequence
Summary: std.file.listdir on OSX produces invalid UTF-8 sequence
Status: RESOLVED FIXED
Alias: None
Product: D
Classification: Unclassified
Component: phobos (show other issues)
Version: D1 (retired)
Hardware: x86 Mac OS X
: P1 critical
Assignee: No Owner
URL:
Keywords:
: 6011 (view as issue list)
Depends on:
Blocks:
Reported: 2009年04月16日 10:42 UTC by Alexandre Fournier
Modified: 2011年05月27日 21:04 UTC (History)
4 users (show)

See Also:


Attachments
Patch that should fix the problem, untested. (782 bytes, patch)
2011年05月15日 15:20 UTC, Jakob Bornecrantz
Details | Diff
Test program from other bug. (167 bytes, text/x-dsrc)
2011年05月15日 15:22 UTC, Jakob Bornecrantz
Details
Add an attachment (proposed patch, testcase, etc.)

Note You need to log in before you can comment on or make changes to this issue.
Description Alexandre Fournier 2009年04月16日 10:42:49 UTC
std.file.listdir throws an "invalid UTF-8 sequence" exception.
please see the source file and the produced output below.
thanks.
------------------------------
import std.stdio, std.file;
int main(char[][] args)
{
	foreach (file; listdir("."))
		writefln("%s", file);	
	return 0;
}
------------------------------
blah:misc user$ dmd listdir.d
blah:misc user$ ./listdir
..
some_directory
Error: 1invalid UTF-8 sequence
blah:misc user$
Comment 1 Mike Kinney 2009年07月16日 23:07:16 UTC
What does the "ls -al" show for that directory? 
Is it happening on all files? 
It works ok on v2.031 when run from the samples dir.
Comment 2 Justin Whear 2009年11月09日 13:15:23 UTC
I can confirm this bug using DMD 1.050 on OS X 10.5.
Running program by OP:
emsi-testings-mac-mini:ratchet emsitesting$ ./listdir 
.
.
ogout
.
e
istory
.
s
.
_central.d
t
re
sion
.
.
ac.sh
inux.sh
_central.o
_central
.
.d
.o
.
The function seems to be trimming the names arbitrarily. Here's ls -al in same directory:
emsi-testings-mac-mini:ratchet emsitesting$ ls -al
total 1324
drwx------ 0 emsitesting staff 16384 Nov 9 13:10 .
drwxrwxrwt@ 5 root admin 170 Nov 5 11:35 ..
-rw--w---- 1 emsitesting staff 6148 Nov 5 11:27 .DS_Store
-rw------- 1 emsitesting staff 7467 Nov 9 12:36 .bash_history
-rw-r--r-- 1 emsitesting staff 220 Nov 4 11:11 .bash_logout
-rw-r--r-- 1 emsitesting staff 3211 Nov 4 13:32 .bashrc
drwxr-xr-x 2 emsitesting staff 16384 Nov 4 13:27 .cache
-rw-r--r-- 1 emsitesting staff 675 Nov 4 11:11 .profile
-rw-r--r-- 1 emsitesting staff 81 Nov 6 12:20 .ratchet
drwx------ 2 emsitesting staff 16384 Nov 4 13:53 .ssh
drwxr-xr-x 3 emsitesting staff 16384 Nov 6 10:50 .subversion
-rwxrwxr-x 1 emsitesting staff 228 Nov 6 13:41 build_linux.sh
-rwxr-xr-x 1 emsitesting staff 187 Nov 6 14:17 build_mac.sh
lrwxrwxrwx 1 emsitesting staff 6 Nov 6 10:50 ccb -> d/ccb/
lrwxrwxrwx 1 emsitesting staff 16 Nov 6 10:54 chains -> d/chains/chains/
drwxr-xr-x 26 emsitesting staff 16384 Nov 9 12:02 d
-rwxr-xr-x 1 emsitesting staff 261192 Nov 9 13:10 listdir
-rw-rw-r-- 1 emsitesting staff 138 Nov 9 13:10 listdir.d
-rw-rw-r-- 1 emsitesting staff 2748 Nov 9 13:10 listdir.o
drwxr-xr-x 6 emsitesting staff 16384 Nov 9 12:34 projects
-rwxr-xr-x 1 emsitesting staff 260223 Nov 6 13:37 ratchet_central
-rw-r--r-- 1 emsitesting staff 2331 Nov 6 13:37 ratchet_central.d
-rw-r--r-- 1 emsitesting staff 9456 Nov 6 13:37 ratchet_central.o
This one's rather critical as I have code that I'm trying to get running on OS X.
(In reply to comment #1)
> What does the "ls -al" show for that directory? 
> 
> Is it happening on all files? 
> 
> It works ok on v2.031 when run from the samples dir.
Comment 3 Justin Whear 2010年08月12日 09:20:03 UTC
This bug still exists in 1.063.
Comment 4 Jakob Bornecrantz 2011年05月15日 14:57:16 UTC
*** Issue 6011 has been marked as a duplicate of this issue. ***
Comment 5 Jakob Bornecrantz 2011年05月15日 15:05:16 UTC
Still exists in 1.064, 1.065 and 1.068.
Comment 6 Jakob Bornecrantz 2011年05月15日 15:19:47 UTC
Looks like the cause of this is due to std.c.linux.linux is being used in std.file where dirent is defined to this:
 struct dirent
 {
 uint d_ino; // this is int on some linuxes
 off_t d_off;
 ushort d_reclen;
 ubyte d_type; // this field isn't there on some linuxes
 char[256] d_name;
 }
While in /usr/include/sys/dirent.h it is defined to this:
struct dirent {
 ino_t d_ino; /* file number of entry */
 __uint16_t d_reclen; /* length of this record */
 __uint8_t d_type; /* file type, see below */
 __uint8_t d_namlen; /* length of string in d_name */
 char d_name[__DARWIN_MAXNAMLEN + 1]; /* name must be no longer than this */
};
Cheers Jakob.
Comment 7 Jakob Bornecrantz 2011年05月15日 15:20:31 UTC
Created attachment 976 [details] 
Patch that should fix the problem, untested.
Comment 8 Jakob Bornecrantz 2011年05月15日 15:22:58 UTC
Created attachment 977 [details] 
Test program from other bug.
Here is the test program from issue 6011.
Cheers Jakob.


AltStyle によって変換されたページ (->オリジナル) /