[Openmcl-devel] pathname-encoding-name

[Gary Byers]

On Sun, 12 Sep 2010, Takehiko Abe wrote:

> ;;; On Linux x86_64
>
> I realized that DIRECTORY won't work with non-ascii filenames
> unless I set PATHNAME-ENCODING-NAME (to :utf-8 in my case).
> This is odd because DIRECTORY worked before without it.

On OSX, all pathnames are encoded in a "normal" form of UTF-8; on
Windows, they're encoded in little-endian UTF-16.  On other Unices,
the filesystem generally just treats them as a sequence of bytes and
higher-level code is supposed to know how they're encoded.  (This is
true of Linux, AFAIK; FreeBSD may have moved toward using UTF-8, and
I'm not sure about Solaris.)  It's not necessary to set
PATHNAME-ENCODING-NAME in order to deal with non-ASCII pathnames on
OSX or Windows (though it's harmless to do so); it is necessary to
do so on other platforms.

>
> Then, I found that RENAME-FILE does not work with non-ascii filenames
> and discovered that CCL::UNIX-RENAME uses plain WITH-CSTRS. This is
> even odder because I think I remember RENAME-FILE worked with Japanese
> filenames before.

I have no recollection of CCL::UNIX-RENAME every having been aware of
non-ASCII (or non-ISO-8859-1) filenames, and it didn't seem to do so
in some of the older releases I looked at.   Of course it should; thanks.

>
> Here's my attempt to fix CCL::UNIX-RENAME for Linux. I left darwin and
> windows version intact but they also wouldn't work for non-latin chars
> as is? And nobody noticed it before? Or am I missing something?

There's a macro called WITH-FILENAME-CSTRs that expands into the right
platform-specific code.  That symbol's exported from CCL; I don't remember
whether or where it was documented, but it's probably the right thing
to use to fix UNIX-RENAME.

Something that I read a few years ago suggested that trying to (do the
equivalent of) set PATHNAME-ENCODING-NAME automatically (from locale
information) was a bad idea, because that locale information was often
wrong.  That might be true, but it might be reasonable to do that (and
suggest that people fix incorrect locale information, which might
affect other programs as well) rather than require that it be set in
one's init file (when it's only really needed/used on some platforms.)

>
>
> diff --git a/source/lib/pathnames.lisp b/source/lib/pathnames.lisp
> index 3dd42d6..886c045 100644
> --- a/source/lib/pathnames.lisp
> +++ b/source/lib/pathnames.lisp
> @@ -78,6 +78,7 @@
> ;-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_-_
> ;File or directory Manipulations
>
> +#+(or darwin-target windows-target)
> (defun unix-rename (old-name new-name)
>   (with-cstrs ((old old-name)
>                (new new-name))
> @@ -89,6 +90,16 @@
>         (values t nil)
>         (values nil (%get-errno))))))
>
> +#-(or darwin-target windows-target)
> +(defun unix-rename (old-name new-name)
> +  (with-encoded-cstrs (pathname-encoding-name) ((old old-name)
> +                                                (new new-name))
> +    (let* ((res (#_rename old new)))
> +      (declare (fixnum res))
> +      (if (zerop res)
> +        (values t nil)
> +        (values nil (%get-errno))))))
> +
> (defun rename-file (file new-name &key (if-exists :error))
>   "Rename FILE to have the specified NEW-NAME. If FILE is a stream open to a
>   file, then the associated file is renamed."
> _______________________________________________
> Openmcl-devel mailing list
> Openmcl-devel at clozure.com
> http://clozure.com/mailman/listinfo/openmcl-devel
>
>