Support MS-Windows file names that use characters outside of ANSI codepage.

 src/w32.c (get_file_security, set_file_security)
 (create_symbolic_link): Separate pointers and boolean flags for
 ANSI and Unicode APIs.  Use the latter if w32_unicode_filenames is
 non-zero, else the former.
 (codepage_for_filenames, filename_to_utf16, )
 (filename_from_utf16, filename_to_ansi, filename_from_ansi): New
 functions.
 (init_user_info): Allow $HOME and $SHELL to include non-ANSI
 characters.
 (normalize_filename): Lose the DBCS code, now works on UTF-8.
 Accept only one argument; all callers changed.
 (dostounix_filename): Remove the second argument, now works in
 UTF-8.  All callers changed.
 (parse_root): Lose DBCS code.
 (get_long_basename, w32_get_short_filename, init_environment)
 (GetCachedVolumeInformation, sys_readdir, open_unc_volume)
 (read_unc_volume, logon_network_drive, faccessat, sys_chdir)
 (sys_chmod, sys_creat, sys_fopen, sys_link, sys_mkdir, sys_open)
 (sys_rename_replace, sys_rmdir, sys_unlink, stat_worker, utime)
 (is_symlink, readlink, chase_symlinks, w32_delayed_load): Work in
 Unicode mode if w32_unicode_filenames is non-zero, in ANSI mode
 otherwise.
 (ansi_encode_filename): New function.
 (get_emacs_configuration, get_emacs_configuration_options):
 Functions deleted.
 (add_volume_info, GetCachedVolumeInformation): Run the input file
 name through unixtodos_filename, to ensure it is stored and
 referenced in canonical form.
 (get_volume_info): Lose the DBCS code, now works in UTF-8.
 (logon_network_drive, sys_link, utime): Improve error handling.
 (sys_access): New function.
 (hashval, generate_inode_val): Unused functions deleted.
 (symlink, readlink, readlinkat): Lose DBCS code, now works in UTF-8.
 (check_windows_init_file): Convert error message from UTF-8 to
 ANSI codepage, for display in the message box.
 (globals_of_w32): Set w32_unicode_filenames according to the OS
 version.
 src/w32term.c (construct_drag_n_drop): Work in Unicode mode when
 w32_unicode_filenames is non-zero, ANSI mode otherwise.
 (syms_of_w32term): Declare w32-unicode-filenames.
 src/w32proc.c (new_child, delete_child): Remove code that handled
 unused pending_deletion and input_file members of the child struct.
 (create_child, sys_spawnve): Convert all file names to ANSI
 codepage.  Use ANSI APIs explicitly; forcibly fail if any file
 name cannot be encoded in ANSI codepage.  Don't use
 unixtodos_filename, mirror slashes by hand.
 (record_infile, record_pending_deletion): Functions deleted.
 (Fw32_short_file_name): Call w32_get_short_filename instead of
 GetShortPathName.
 src/w32notify.c (add_watch): Work in Unicode mode when
 w32_unicode_filenames is non-zero, ANSI mode otherwise.
 (Fw32notify_add_watch): Rewrite to avoid using GetFullPathName;
 instead, do the same with Lisp primitives.
 src/w32fns.c (file_dialog_callback, Fx_file_dialog)
 (Fsystem_move_file_to_trash, Fw32_shell_execute)
 (Ffile_system_info, Fdefault_printer_name): Work in Unicode mode
 when w32_unicode_filenames is non-zero, ANSI mode otherwise.
 (Fw32_shell_execute): Improve error reporting.
 (Fdefault_printer_name): Ifdef away for Cygwin.
 src/w32.h (struct _child_process): Remove input_file and
 pending_deletion members that are no longer used.
 (dostounix_filename, w32_get_short_filename, filename_from_ansi)
 (filename_to_ansi, filename_from_utf16, filename_to_utf16)
 (ansi_encode_filename): New and updated prototypes.
 src/unexw32.c (open_input_file, open_output_file, unexec): Use ANSI
 APIs explicitly.
 (unexec): Don't use dostounix_filename, it expects a file name in
 UTF-8.  Instead, mirror backslashes by hand.  Convert NEW_NAME to
 ANSI encoding.
 src/fileio.c (Ffile_name_directory, file_name_as_directory)
 (directory_file_name, Fexpand_file_name)
 (Fsubstitute_in_file_name) [WINDOWSNT]: Adapt to the change in
 arguments of dostounix_filename.
 (Fexpand_file_name) [WINDOWSNT]: Convert value of $HOME to UTF-8.
 use MAX_UTF8_PATH for size of file-name strings.
 (emacs_readlinkat): Build an explicitly unibyte string for file
 names.
 (syms_of_fileio) <file-name-coding-system>
 default-file-name-coding-system>: Mention MS-Windows peculiarities.
 src/emacs.c (init_cmdargs) [WINDOWSNT]: Convert argv[0] to UTF-8.
 (main) [WINDOWSNT]: Convert the argv[] elements that are files or
 directories to UTF-8.
 (decode_env_path) [WINDOWSNT]: Convert file names taken from the
 environment, and each element of the input PATH, to UTF-8.
 src/dired.c (file_attributes): Use build_unibyte_string explicitly
 to make Lisp strings from user and group names.
 src/coding.h (ENCODE_FILE, DECODE_FILE): Just call encode_file and
 decode_file.
 src/coding.c (decode_file_name, encode_file_name): New functions.
 src/termcap.c (tgetent): Adapt to the change in arguments of
 dostounix_filename.
 src/sysdep.c (sys_subshell) [WINDOWSNT]: Use MAX_UTF8_PATH for file
 names.
 src/msdos.c (dostounix_filename, init_environment): Adapt to the
 change in arguments of dostounix_filename.
 src/image.c (xpm_load, tiff_load, gif_load, imagemagick_load)
 [WINDOWSNT]: Encode file names passed to the image libraries in
 ANSI codepage.
 src/gnutls.c (Fgnutls_boot): Encode all file names passed to GnuTLS.
 [WINDOWSNT]: Convert file names to the current ANSI codepage.
 src/filelock.c (lock_file) [WINDOWSNT]: Adapt to the change in
 arguments of dostounix_filename.

 nt/inc/ms-w32.h (MAX_UTF8_PATH): New macro.
 (opendir, closedir, readdir, seekdir): Redirect to replacement
 functions.
 nt/inc/dirent.h: Make d_name[] be MAXNAMELEN*4 characters long.

 lisp/term/w32-win.el (w32-handle-dropped-file):
 lisp/startup.el (normal-top-level):
 lisp/net/browse-url.el (browse-url-file-url):
 lisp/dnd.el (dnd-get-local-file-name): On MS-Windows, encode and
 decode file names using 'utf-8' rather than
 file-name-coding-system.

 doc/emacs/mule.texi (File Name Coding): Document file-name encoding
 peculiarities on MS-Windows.

 doc/lispref/nonascii.texi (Encoding and I/O): Document file-name encoding
 peculiarities on MS-Windows.

 etc/NEWS: Mention support on MS-Windows of file names outside of the
 current locale.

Fixes: debbugs:7100

5 files changed, 75 insertions, 52 deletions

diff --git a/lisp/ChangeLog b/lisp/ChangeLog
index 52d4eff3eb6..c761bce7518 100644
--- a/lisp/ChangeLog
+++ b/lisp/ChangeLog
@@ -1,3 +1,12 @@
+2013-12-12  Eli Zaretskii  <eliz@gnu.org>
+
+	* term/w32-win.el (w32-handle-dropped-file):
+	* startup.el (normal-top-level):
+	* net/browse-url.el (browse-url-file-url):
+	* dnd.el (dnd-get-local-file-name): On MS-Windows, encode and
+	decode file names using 'utf-8' rather than
+	file-name-coding-system.
+
 2013-12-12  Fabián Ezequiel Gallina  <fgallina@gnu.org>
 
 	* progmodes/python.el (python-indent-context)
diff --git a/lisp/dnd.el b/lisp/dnd.el
index d9061273c32..a4aa4f4558e 100644
--- a/lisp/dnd.el
+++ b/lisp/dnd.el
@@ -152,10 +152,13 @@ Return nil if URI is not a local file."
   (let ((f (cond ((string-match "^file:///" uri)	; XDND format.
 		  (substring uri (1- (match-end 0))))
 		 ((string-match "^file:" uri)		; Old KDE, Motif, Sun
-		  (substring uri (match-end 0))))))
-    (and f (setq f (decode-coding-string (dnd-unescape-uri f)
-                                         (or file-name-coding-system
-                                             default-file-name-coding-system))))
+		  (substring uri (match-end 0)))))
+	(coding (if (equal system-type 'windows-nt)
+		    ;; W32 pretends that file names are UTF-8 encoded.
+		    'utf-8
+		  (or file-name-coding-system
+		      default-file-name-coding-system))))
+    (and f (setq f (decode-coding-string (dnd-unescape-uri f) coding)))
     (when (and f must-exist (not (file-readable-p f)))
       (setq f nil))
     f))
diff --git a/lisp/net/browse-url.el b/lisp/net/browse-url.el
index ff654f23747..461a0543829 100644
--- a/lisp/net/browse-url.el
+++ b/lisp/net/browse-url.el
@@ -723,9 +723,12 @@ interactively.  Turn the filename into a URL with function
 (defun browse-url-file-url (file)
   "Return the URL corresponding to FILE.
 Use variable `browse-url-filename-alist' to map filenames to URLs."
-  (let ((coding (and (default-value 'enable-multibyte-characters)
-		     (or file-name-coding-system
-			 default-file-name-coding-system))))
+  (let ((coding (if (equal system-type 'windows-nt)
+		    ;; W32 pretends that file names are UTF-8 encoded.
+		    'utf-8
+		  (and (default-value 'enable-multibyte-characters)
+		       (or file-name-coding-system
+			   default-file-name-coding-system)))))
     (if coding (setq file (encode-coding-string file coding))))
   (setq file (browse-url-url-encode-chars file "[*\"()',=;?% ]"))
   (dolist (map browse-url-filename-alist)
diff --git a/lisp/startup.el b/lisp/startup.el
index a21695fe19a..bbcf5bda2d7 100644
--- a/lisp/startup.el
+++ b/lisp/startup.el
@@ -533,43 +533,45 @@ It is the default value of the variable `top-level'."
     ;; for many other file-name variables and directory lists, so it
     ;; is important to decode it ASAP.
     (when locale-coding-system
-      (save-excursion
-	(dolist (elt (buffer-list))
-	  (set-buffer elt)
-	  (if default-directory
-	      (setq default-directory
-		    (decode-coding-string default-directory
-					  locale-coding-system t)))))
-
-      ;; Decode all the important variables and directory lists, now
-      ;; that we know the locale's encoding.  This is because the
-      ;; values of these variables are until here unibyte undecoded
-      ;; strings created by build_unibyte_string.  data-directory in
-      ;; particular is used to construct many other standard directory
-      ;; names, so it must be decoded ASAP.
-      ;; Note that charset-map-path cannot be decoded here, since we
-      ;; could then be trapped in infinite recursion below, when we
-      ;; load subdirs.el, because encoding a directory name might need
-      ;; to load a charset map, which will want to encode
-      ;; charset-map-path, which will want to load the same charset
-      ;; map...  So decoding of charset-map-path is delayed until
-      ;; further down below.
-      (dolist (pathsym '(load-path exec-path))
-	(let ((path (symbol-value pathsym)))
-	  (if (listp path)
-	      (set pathsym (mapcar (lambda (dir)
-				     (decode-coding-string
-				      dir
-				      locale-coding-system t))
-				path)))))
-      (dolist (filesym '(data-directory doc-directory exec-directory
-					installation-directory
-					invocation-directory invocation-name
-					source-directory
-					shared-game-score-directory))
-	(let ((file (symbol-value filesym)))
-	  (if (stringp file)
-	      (set filesym (decode-coding-string file locale-coding-system t))))))
+      (let ((coding (if (eq system-type 'windows-nt)
+			;; MS-Windows build converts all file names to
+			;; UTF-8 during startup.
+			'utf-8
+		      locale-coding-system)))
+	(save-excursion
+	  (dolist (elt (buffer-list))
+	    (set-buffer elt)
+	    (if default-directory
+		(setq default-directory
+		      (decode-coding-string default-directory coding t)))))
+
+	;; Decode all the important variables and directory lists, now
+	;; that we know the locale's encoding.  This is because the
+	;; values of these variables are until here unibyte undecoded
+	;; strings created by build_unibyte_string.  data-directory in
+	;; particular is used to construct many other standard
+	;; directory names, so it must be decoded ASAP.  Note that
+	;; charset-map-path cannot be decoded here, since we could
+	;; then be trapped in infinite recursion below, when we load
+	;; subdirs.el, because encoding a directory name might need to
+	;; load a charset map, which will want to encode
+	;; charset-map-path, which will want to load the same charset
+	;; map...  So decoding of charset-map-path is delayed until
+	;; further down below.
+	(dolist (pathsym '(load-path exec-path))
+	  (let ((path (symbol-value pathsym)))
+	    (if (listp path)
+		(set pathsym (mapcar (lambda (dir)
+				       (decode-coding-string dir coding t))
+				     path)))))
+	(dolist (filesym '(data-directory doc-directory exec-directory
+					  installation-directory
+					  invocation-directory invocation-name
+					  source-directory
+					  shared-game-score-directory))
+	  (let ((file (symbol-value filesym)))
+	    (if (stringp file)
+		(set filesym (decode-coding-string file coding t)))))))
 
     (let ((dir default-directory))
       (with-current-buffer "*Messages*"
@@ -599,12 +601,13 @@ It is the default value of the variable `top-level'."
     ;; need for encoding them are already loaded, we are ready to
     ;; decode charset-map-path.
     (if (listp charset-map-path)
-	(setq charset-map-path
-	      (mapcar (lambda (dir)
-			(decode-coding-string
-			 dir
-			 locale-coding-system t))
-		      charset-map-path)))
+	(let ((coding (if (eq system-type 'windows-nt)
+			  'utf-8
+			locale-coding-system)))
+	  (setq charset-map-path
+		(mapcar (lambda (dir)
+			  (decode-coding-string dir coding t))
+			charset-map-path))))
     (setq default-directory (abbreviate-file-name default-directory))
     (let ((old-face-font-rescale-alist face-font-rescale-alist))
       (unwind-protect
diff --git a/lisp/term/w32-win.el b/lisp/term/w32-win.el
index 513a99d4914..af1a351c0ad 100644
--- a/lisp/term/w32-win.el
+++ b/lisp/term/w32-win.el
@@ -110,8 +110,13 @@
   (let ((f (if (eq system-type 'cygwin)
                (cygwin-convert-file-name-from-windows file-name t)
              (subst-char-in-string ?\\ ?/ file-name)))
-        (coding (or file-name-coding-system
-                    default-file-name-coding-system)))
+        (coding (if (eq system-type 'windows-nt)
+		    ;; Native w32 build pretends that its file names
+		    ;; are encoded in UTF-8, and converts to the
+		    ;; appropriate encoding internally.
+		    'utf-8
+		  (or file-name-coding-system
+		      default-file-name-coding-system))))
 
     (setq file-name
           (mapconcat 'url-hexify-string