From amol.vinayak.deshpande at gmail.com  Wed Dec  9 00:01:57 2009
From: amol.vinayak.deshpande at gmail.com (Amol Deshpande)
Date: Tue, 8 Dec 2009 14:01:57 -0800
Subject: fix for IGNORE macro and associated function in sh.h
Message-ID: <8158a9230912081401t601e703es925a97fef023d8ee@mail.gmail.com>

That code does not compile with Visual C++. IGNORE is already defined and
something about "inline" in the function definition upsets the compiler. I
don't see any harm in removing the inline attribute, since the compiler
should be smart enough about optimizing it out anyway.

Please let me know if this is OK to check in.

thanks,
-amol


*** sh.h.orig Thu Jun 25 13:27:37 2009
--- sh.h Tue Dec 08 13:56:14 2009
***************
*** 116,123 ****

  /* Elide unused argument warnings */
  #define USE(a) (void) (a)
! #define IGNORE(a) ignore((intptr_t)a)
! static inline void ignore(intptr_t a)
  {
      USE(a);
  }
--- 116,123 ----

  /* Elide unused argument warnings */
  #define USE(a) (void) (a)
! #define TCSH_IGNORE(a) ignore((intptr_t)a)
! static /*inline*/ void ignore(intptr_t a)
  {
      USE(a);
  }
*** sh.sem.c.orig Thu Jun 25 13:27:38 2009
--- sh.sem.c Tue Dec 08 13:57:10 2009
***************
*** 840,846 ****
   }
   else if (flags & F_PIPEIN) {
       xclose(0);
!      IGNORE(dup(pipein[0]));
       xclose(pipein[0]);
       xclose(pipein[1]);
   }
--- 840,846 ----
   }
   else if (flags & F_PIPEIN) {
       xclose(0);
!      TCSH_IGNORE(dup(pipein[0]));
       xclose(pipein[0]);
       xclose(pipein[1]);
   }
***************
*** 850,856 ****
   }
   else {
       xclose(0);
!      IGNORE(dup(OLDSTD));
  #if defined(CLOSE_ON_EXEC) && defined(CLEX_DUPS)
       /*
        * PWP: Unlike Bezerkeley 4.3, FIONCLEX for Pyramid is preserved
--- 850,856 ----
   }
   else {
       xclose(0);
!      TCSH_IGNORE(dup(OLDSTD));
  #if defined(CLOSE_ON_EXEC) && defined(CLEX_DUPS)
       /*
        * PWP: Unlike Bezerkeley 4.3, FIONCLEX for Pyramid is preserved
***************
*** 903,914 ****
      }
      else if (flags & F_PIPEOUT) {
   xclose(1);
!  IGNORE(dup(pipeout[1]));
   is1atty = 0;
      }
      else {
   xclose(1);
!  IGNORE(dup(SHOUT));
   is1atty = isoutatty;
  # if defined(CLOSE_ON_EXEC) && defined(CLEX_DUPS)
   (void) close_on_exec(1, 0);
--- 903,914 ----
      }
      else if (flags & F_PIPEOUT) {
   xclose(1);
!  TCSH_IGNORE(dup(pipeout[1]));
   is1atty = 0;
      }
      else {
   xclose(1);
!  TCSH_IGNORE(dup(SHOUT));
   is1atty = isoutatty;
  # if defined(CLOSE_ON_EXEC) && defined(CLEX_DUPS)
   (void) close_on_exec(1, 0);
***************
*** 917,927 ****

      xclose(2);
      if (flags & F_STDERR) {
!  IGNORE(dup(1));
   is2atty = is1atty;
      }
      else {
!  IGNORE(dup(SHDIAG));
   is2atty = isdiagatty;
  # if defined(CLOSE_ON_EXEC) && defined(CLEX_DUPS)
   (void) close_on_exec(2, 0);
--- 917,927 ----

      xclose(2);
      if (flags & F_STDERR) {
!  TCSH_IGNORE(dup(1));
   is2atty = is1atty;
      }
      else {
!  TCSH_IGNORE(dup(SHDIAG));
   is2atty = isdiagatty;
  # if defined(CLOSE_ON_EXEC) && defined(CLEX_DUPS)
   (void) close_on_exec(2, 0);

From christos at zoulas.com  Wed Dec  9 06:47:27 2009
From: christos at zoulas.com (Christos Zoulas)
Date: Tue, 8 Dec 2009 23:47:27 -0500
Subject: fix for IGNORE macro and associated function in sh.h
In-Reply-To: <8158a9230912081401t601e703es925a97fef023d8ee@mail.gmail.com>
	from Amol Deshpande (Dec  8,  2:01pm)
Message-ID: <20091209044727.389C85654E@rebar.astron.com>

On Dec 8,  2:01pm, amol.vinayak.deshpande at gmail.com (Amol Deshpande) wrote:
-- Subject: fix for IGNORE macro and associated function in sh.h

| That code does not compile with Visual C++. IGNORE is already defined and
| something about "inline" in the function definition upsets the compiler. I
| don't see any harm in removing the inline attribute, since the compiler
| should be smart enough about optimizing it out anyway.
| 
| Please let me know if this is OK to check in.

Go for it.

Thanks,

christos


From vinschen at redhat.com  Sat Dec 26 22:29:58 2009
From: vinschen at redhat.com (Corinna Vinschen)
Date: Sat, 26 Dec 2009 21:29:58 +0100
Subject: Correct handling of wide chars on systems with sizeof(wchar_t)==2?
Message-ID: <20091226202958.GB27407@calimero.vinschen.de>

Hi,

the subject sounds a bit long.  What I'm trying to ask is this.

Now that the new Cygwin 1.7 has full multibyte support it turns out that
tcsh doesn't work correctly with multibyte charsets.  The reason is that
WIDE_STRING support only works if sizeof(wchar_t) is at least 4.

Unfortunately sizeof(wchar_t) on Windows is 2, so, even though Cygwin
can handle UTF-8 just fine, tcsh on Cygwin can't.  And there's also no
way at all to fix that in tcsh since tcsh uses the (IMHO) ugly method to
use unused bits in a character to classify the character lexically, see
sh,h, line 699ff.  That method only works for wchar_t == UTF-32 but not
for wchar_t == UTF-16.

My question is this:  Is there any effort on the way to allow correct
WIDE_STRING support also for systems like Cygwin?  If not, is there any
chance we can discuss how to change tcsh to make this work?


Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat


From vinschen at redhat.com  Sat Dec 26 23:01:41 2009
From: vinschen at redhat.com (Corinna Vinschen)
Date: Sat, 26 Dec 2009 22:01:41 +0100
Subject: [PATCH] build error on systems not having NLS_CATALOGS defined
Message-ID: <20091226210141.GC27407@calimero.vinschen.de>

Hi,

the following patch is necessary on systems not defining NLS_CATALOGS,
otherwise sh.c can't be compiled.

The reason is that STRNLSPATH is only defined on systems having
NLS_CATALOGS defined as well.  And, btw., the name of the
add_localedir_to_nslpath function seems to have a typo.  Shouldn't that
be add_localedir_to_nlspath
                    ^^^
                    rather than nsl?


Index: sh.c
===================================================================
RCS file: /p/tcsh/cvsroot/tcsh/sh.c,v
retrieving revision 3.150
diff -u -p -r3.150 sh.c
--- sh.c	17 Oct 2009 17:45:47 -0000	3.150
+++ sh.c	26 Dec 2009 20:59:23 -0000
@@ -164,6 +164,7 @@ static	void		  st_restore	(void *);
 #define LOCALEDIR "/usr/share/locale"
 #endif
 
+#ifdef NLS_CATALOGS
 static void
 add_localedir_to_nslpath(const char *path)
 {
@@ -193,6 +194,7 @@ add_localedir_to_nslpath(const char *pat
     tsetenv(STRNLSPATH, str2short(new));
     free(new);
 }
+#endif
 
 int
 main(int argc, char **argv)
@@ -229,7 +231,9 @@ main(int argc, char **argv)
     STR_environ = blk2short(environ);
     environ = short2blk(STR_environ);	/* So that we can free it */
 
+#ifdef NLS_CATALOGS
     add_localedir_to_nslpath(LOCALEDIR);
+#endif
 
     nlsinit();
 

Thanks,
Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat


From christos at zoulas.com  Sat Dec 26 23:40:29 2009
From: christos at zoulas.com (Christos Zoulas)
Date: Sat, 26 Dec 2009 16:40:29 -0500
Subject: Correct handling of wide chars on systems with sizeof(wchar_t)==2?
In-Reply-To: <20091226202958.GB27407@calimero.vinschen.de>
	from Corinna Vinschen (Dec 26,  9:29pm)
Message-ID: <20091226214030.12FFD5654E@rebar.astron.com>

On Dec 26,  9:29pm, vinschen at redhat.com (Corinna Vinschen) wrote:
-- Subject: Correct handling of wide chars on systems with sizeof(wchar_t)==2

| Hi,
| 
| the subject sounds a bit long.  What I'm trying to ask is this.
| 
| Now that the new Cygwin 1.7 has full multibyte support it turns out that
| tcsh doesn't work correctly with multibyte charsets.  The reason is that
| WIDE_STRING support only works if sizeof(wchar_t) is at least 4.
| 
| Unfortunately sizeof(wchar_t) on Windows is 2, so, even though Cygwin
| can handle UTF-8 just fine, tcsh on Cygwin can't.  And there's also no
| way at all to fix that in tcsh since tcsh uses the (IMHO) ugly method to
| use unused bits in a character to classify the character lexically, see
| sh,h, line 699ff.  That method only works for wchar_t == UTF-32 but not
| for wchar_t == UTF-16.
| 
| My question is this:  Is there any effort on the way to allow correct
| WIDE_STRING support also for systems like Cygwin?  If not, is there any
| chance we can discuss how to change tcsh to make this work?

Yes, we should detect another case where sizeof(wchar_t) == 2 and then
make Char int32_t, and deal with converting back and forth between Char
and wchar_t the same way we used to convert when using short string 
(between Char (int16_t) and char).

christos


From vinschen at redhat.com  Sun Dec 27 14:40:37 2009
From: vinschen at redhat.com (Corinna Vinschen)
Date: Sun, 27 Dec 2009 13:40:37 +0100
Subject: Correct handling of wide chars on systems with sizeof(wchar_t)==2?
In-Reply-To: <20091226214030.12FFD5654E@rebar.astron.com>
References: <20091226202958.GB27407@calimero.vinschen.de>
	<20091226214030.12FFD5654E@rebar.astron.com>
Message-ID: <20091227124036.GD27407@calimero.vinschen.de>

On Dec 26 16:40, Christos Zoulas wrote:
> On Dec 26,  9:29pm, vinschen at redhat.com (Corinna Vinschen) wrote:
> -- Subject: Correct handling of wide chars on systems with sizeof(wchar_t)==2
> 
> | Hi,
> | 
> | the subject sounds a bit long.  What I'm trying to ask is this.
> | 
> | Now that the new Cygwin 1.7 has full multibyte support it turns out that
> | tcsh doesn't work correctly with multibyte charsets.  The reason is that
> | WIDE_STRING support only works if sizeof(wchar_t) is at least 4.
> | 
> | Unfortunately sizeof(wchar_t) on Windows is 2, so, even though Cygwin
> | can handle UTF-8 just fine, tcsh on Cygwin can't.  And there's also no
> | way at all to fix that in tcsh since tcsh uses the (IMHO) ugly method to
> | use unused bits in a character to classify the character lexically, see
> | sh,h, line 699ff.  That method only works for wchar_t == UTF-32 but not
> | for wchar_t == UTF-16.
> | 
> | My question is this:  Is there any effort on the way to allow correct
> | WIDE_STRING support also for systems like Cygwin?  If not, is there any
> | chance we can discuss how to change tcsh to make this work?
> 
> Yes, we should detect another case where sizeof(wchar_t) == 2 and then
> make Char int32_t, and deal with converting back and forth between Char
> and wchar_t the same way we used to convert when using short string 
> (between Char (int16_t) and char).

Do we really need another case?  Wouldn't it be sufficient to redefine
SHORT_STRINGS to be that case?  It might be helpful to define Char as
wint_t in this case.  wint_t is typically defined as 4 byte unsigned on
a UTF-16 system anyway, otherwise there's no way to define WEOF.


Corinna

-- 
Corinna Vinschen
Cygwin Project Co-Leader
Red Hat


From christos at zoulas.com  Sun Dec 27 18:32:46 2009
From: christos at zoulas.com (Christos Zoulas)
Date: Sun, 27 Dec 2009 11:32:46 -0500
Subject: Correct handling of wide chars on systems with sizeof(wchar_t)==2?
In-Reply-To: <20091227124036.GD27407@calimero.vinschen.de>
	from Corinna Vinschen (Dec 27,  1:40pm)
Message-ID: <20091227163246.A40C15654F@rebar.astron.com>

On Dec 27,  1:40pm, vinschen at redhat.com (Corinna Vinschen) wrote:
-- Subject: Re: Correct handling of wide chars on systems with sizeof(wchar_t

| > Yes, we should detect another case where sizeof(wchar_t) == 2 and then
| > make Char int32_t, and deal with converting back and forth between Char
| > and wchar_t the same way we used to convert when using short string 
| > (between Char (int16_t) and char).
| 
| Do we really need another case?  Wouldn't it be sufficient to redefine
| SHORT_STRINGS to be that case?  It might be helpful to define Char as
| wint_t in this case.  wint_t is typically defined as 4 byte unsigned on
| a UTF-16 system anyway, otherwise there's no way to define WEOF.

It might be simpler, I don't know.

christos