Thread Local Linking

I have a legacy project with hundreds of unmanaged C programs, that I am
porting to .NET. There are a lot of globally scoped variables in the C
programs, which I have had to convert to thread local, so that multiple
threads of the same program do not interfere with each other. The syntax is
like this:

system.h contains:
#define Thread __declspec(thread)

prog1.c contains:
Thread int abc;

prog2.c contains:
Thread extern int abc;

That all works fine, when all the references to the variable have the Thread
prefix. The problem is when something is missing, and it is like this:

prog1.c contains:
Thread int abc;

prog2.c contains:
extern int abc;

In this case, the compiler treats them as totally separate variables, with
separate storage, which causes the program to crash in hard-to-fix ways. I
had expected that the linker would consider the extern int abc; to be an
unresolved external, since it considers it different than the Thread int
abc;, but it does not. It links with no error or warning, then crashes when
run. It will be difficult and unreliable for me to find all the instances of
this, I was hoping that they was a way of getting the linker to detect these,
or a way of making the extern int abc; be made to refer to the Thread int,
without having to find each instance in the source code. I appreciate any
suggestions anyone has.

RE: Thread Local Linking

> It will be difficult and unreliable for me to find all the instances of
> this, I was hoping that they was a way of getting the linker to detect these,
> or a way of making the extern int abc; be made to refer to the Thread int,
> without having to find each instance in the source code.

1. Maybe you could use the compiler for assistance rather than the linker.
You could replace all 'extern' with 'EXTERN'. Then replace all 'Thread
EXTERN' with 'Thread extern'. The remaining EXTERNs are your problem area,
and the compiler will find them all. If it is a certainty that every
remaining EXTERN should be Thread extern, then you can change them all
without using the compiler at all. But I would guess that not all extern's
should be Thread extern.

2. When you said "hundreds of unmanaged C programs", I assume you meant
hundreds of .c files. You might consider collapsing several related files
into one file containing several C functions. If done adroitly, many
extern's will disappear.

RE: Thread Local Linking

Thanks for the suggestions. Yes, there are about 1,500 C program files. I
have done what I can in terms of combining programs to eliminate externals,
but it is a multi-programmer production environment, if I combine them too
much we can't work because large portions are booked to someone to work on.

Yes, as you say, the problem is that some of them must be thread extern, but
some can not be. I wrote a program to parse the source code and convert, but
there are cases that it did incorrectly, where a line was split, or there was
a misleading comment, or something like that, it is those that I am trying to
find.

Thanks again for the assistance, and if anyone knows a way to make the
linker detect this, I would appreciate hearing about it.


"AMercer" wrote:

> > It will be difficult and unreliable for me to find all the instances of
> > this, I was hoping that they was a way of getting the linker to detect these,
> > or a way of making the extern int abc; be made to refer to the Thread int,
> > without having to find each instance in the source code.
>
> 1. Maybe you could use the compiler for assistance rather than the linker.
> You could replace all 'extern' with 'EXTERN'. Then replace all 'Thread
> EXTERN' with 'Thread extern'. The remaining EXTERNs are your problem area,
> and the compiler will find them all. If it is a certainty that every
> remaining EXTERN should be Thread extern, then you can change them all
> without using the compiler at all. But I would guess that not all extern's
> should be Thread extern.
>
> 2. When you said "hundreds of unmanaged C programs", I assume you meant
> hundreds of .c files. You might consider collapsing several related files
> into one file containing several C functions. If done adroitly, many
> extern's will disappear.
>

RE: Thread Local Linking

Hi Richard,

This does look like an issue of the Visual C++ compiler as the VS7.1
version of MSDN documentation stated that it's an error to define like this:

#thread (C++)
http://msdn2.microsoft.com/en-us/library/9w1sdazb(VS.71).asp x

You must use the thread attribute for the declaration and the definition of
a thread local object, whether the declaration and definition occur in the
same file or separate files. For example, the following code generates an
error:

#define Thread __declspec( thread )
extern int tls_i; // This generates an error, because the
int Thread tls_i; // declaration and the definition differ.



I'm currently consulting within our internal discussion list to get more
information on this. I will keep you posted.

By the way, which version of Visual Studio are you using?


Regards,
Walter Wang (wawang@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================

This posting is provided "AS IS" with no warranties, and confers no rights.

RE: Thread Local Linking

Thanks Walter. Yes, I get that error when I have the conflicting definitions
in the same file, but when they are in different files it gets no error, the
variable just ends up undefined and crashes at runtime. I am building with
the command line exclusively, it is cl.exe version 14.00.50727.42. Thanks.


""Walter Wang [MSFT]"" wrote:

> Hi Richard,
>
> This does look like an issue of the Visual C++ compiler as the VS7.1
> version of MSDN documentation stated that it's an error to define like this:
>
> #thread (C++)
> http://msdn2.microsoft.com/en-us/library/9w1sdazb(VS.71).asp x
>
> You must use the thread attribute for the declaration and the definition of
> a thread local object, whether the declaration and definition occur in the
> same file or separate files. For example, the following code generates an
> error:
>
> #define Thread __declspec( thread )
> extern int tls_i; // This generates an error, because the
> int Thread tls_i; // declaration and the definition differ.
>
>
>
> I'm currently consulting within our internal discussion list to get more
> information on this. I will keep you posted.
>
> By the way, which version of Visual Studio are you using?
>
>
> Regards,
> Walter Wang (wawang@online.microsoft.com, remove 'online.')
> Microsoft Online Community Support
>
> ==================================================
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ==================================================
>
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
>

RE: Thread Local Linking

Hi Richard,

We have discussed this issue internally, it's still hard to say whether or
not it's a by design behavior or issue in the compiler/linker.

Unfortunately we don't see if there's any good workaround for your scenario
since you have a large code base that are already using the "extern"
declaration whenever it's needed. We are really sorry for the inconvenience
caused.

On the other hand, if we were using separate header files to declare those
extern global variables, then we might be able to use following workaround
to make the compiler find such issue for us. For example:

// a.h
extern int abc;

// a.c
#include "a.h"
void main()
{
abc = 1;
}

// b.c
#include "a.h"
__declspec(thread) int abc;


Regards,
Walter Wang (wawang@online.microsoft.com, remove 'online.')
Microsoft Online Community Support

==================================================
When responding to posts, please "Reply to Group" via your newsreader so
that others may learn and benefit from your issue.
==================================================

This posting is provided "AS IS" with no warranties, and confers no rights.

RE: Thread Local Linking

Thanks Walter, I had somewhat come to that conclusion myself. I don't see how
anyone could consider it intended behavior, since it is harmful and has no
advantage, but it is what it is. Yes, your solution with a header file is
useful, I have been going through the code and cleaning up all the different
places where things are defined. Thanks again for the help.

""Walter Wang [MSFT]"" wrote:

> Hi Richard,
>
> We have discussed this issue internally, it's still hard to say whether or
> not it's a by design behavior or issue in the compiler/linker.
>
> Unfortunately we don't see if there's any good workaround for your scenario
> since you have a large code base that are already using the "extern"
> declaration whenever it's needed. We are really sorry for the inconvenience
> caused.
>
> On the other hand, if we were using separate header files to declare those
> extern global variables, then we might be able to use following workaround
> to make the compiler find such issue for us. For example:
>
> // a.h
> extern int abc;
>
> // a.c
> #include "a.h"
> void main()
> {
> abc = 1;
> }
>
> // b.c
> #include "a.h"
> __declspec(thread) int abc;
>
>
> Regards,
> Walter Wang (wawang@online.microsoft.com, remove 'online.')
> Microsoft Online Community Support
>
> ==================================================
> When responding to posts, please "Reply to Group" via your newsreader so
> that others may learn and benefit from your issue.
> ==================================================
>
> This posting is provided "AS IS" with no warranties, and confers no rights.
>
>