|
|
 | | From: | Jim Rogers | | Subject: | Re: Synchronization Vs Concurrency | | Date: | Wed, 17 Nov 2004 00:04:12 GMT |
|
|
 | "Zahid Faizal" wrote in
news:1100639531.519829.256120@c13g2000cwb.googlegroups.com:
> I have often heard about synchronization and concurrency being talked
> in tandem. I kinda know what this is all about ----- semaphores,
> controlled access to critical resources blah blah blah. However, I am
> not sure of the delineation between the two. What is the
> "synchronization" part in this, and what is the "concurrency" part?
> Can the two be talked about in isolation, or do they always exist in
> tandem?
>
> Any insight will be appreciated.
(I have trimmed the massive cross postings)
Synchronization is one tool used in concurrent programs to ensure
data integrity for data shared by several concurrent tasks.
Concurrent programs can be written without any form of
synchronization. In that case, either the concurrent tasks share
no data, or any data corruption due to unsynchronized sharing
is either impossible or unimportant.
Concurrency deals with the creation, scheduling, and lifetime
of concurrent tasks. Synchronization deals with the protected
sharing of data and state information between concurrent tasks.
A single-threaded program is completely synchronous.
The purpose behind multi-threaded applications is to achieve
greater asynchronous behavior, allowing many things to be
done at the same time. Many multi-threaded programs still need
some degree of synchronization to facilitate reliable communcation
between concurrent tasks.
Asynchronous communication often takes the form of signals.
All the operating systems I am familiar with provide some set of
signals to indicate a desired change of state in the concurrent
task receiving the signal. Other than the signal itself, no data
is sent from one concurrent task to the other in the communication
of a signal. This is the reason that signals are very limited in
their applicability as a communication between concurrent tasks.
Most operating systems only support signals at the process level,
and not at the thread level. Linux is an exception because of the
strong similarity between threads and processes.
Jim Rogers
|
|
| | From: | John Morrison | | Subject: | Re: Synchronization Vs Concurrency | | Date: | Sun, 19 Dec 2004 04:34:40 -0000 |
|
|
| Jim Rogers wrote:
> "Zahid Faizal" wrote in
> news:1100639531.519829.256120@c13g2000cwb.googlegroups.com:
>
>> I have often heard about synchronization and concurrency being talked
>> in tandem. I kinda know what this is all about ----- semaphores,
>> controlled access to critical resources blah blah blah. However, I
>> am not sure of the delineation between the two. What is the
>> "synchronization" part in this, and what is the "concurrency" part?
>> Can the two be talked about in isolation, or do they always exist in
>> tandem?
>>
>> Any insight will be appreciated.
>
> (I have trimmed the massive cross postings)
>
> Synchronization is one tool used in concurrent programs to ensure
> data integrity for data shared by several concurrent tasks.
>
> Concurrent programs can be written without any form of
> synchronization. In that case, either the concurrent tasks share
> no data, or any data corruption due to unsynchronized sharing
> is either impossible or unimportant.
>
> Concurrency deals with the creation, scheduling, and lifetime
> of concurrent tasks. Synchronization deals with the protected
> sharing of data and state information between concurrent tasks.
>
> A single-threaded program is completely synchronous.
> The purpose behind multi-threaded applications is to achieve
> greater asynchronous behavior, allowing many things to be
> done at the same time. Many multi-threaded programs still need
> some degree of synchronization to facilitate reliable communcation
> between concurrent tasks.
>
> Asynchronous communication often takes the form of signals.
> All the operating systems I am familiar with provide some set of
> signals to indicate a desired change of state in the concurrent
> task receiving the signal. Other than the signal itself, no data
> is sent from one concurrent task to the other in the communication
> of a signal. This is the reason that signals are very limited in
> their applicability as a communication between concurrent tasks.
> Most operating systems only support signals at the process level,
> and not at the thread level. Linux is an exception because of the
> strong similarity between threads and processes.
>
> Jim Rogers
The only other thing which needs emphasis here is real *vs* apparent
concurrency. The former is available (indeed, inevitable!) on systems with
more than one processor, and needs some "tricks" (/e.g./ spin locks) at the
system level for guarding OS resources against simultaneous access by OS
code on different processors. With one processor only, all concurrency is
apparent, since one processor can - obviously! - do only one thing at a
time.
At the process/thread level, there should be no difference: even if two
(or more) threads of the same process are running simultaneously on
different processors, the OS's mutual exclusion provisions - whatever names
they have on the OS you are using - are used in exactly the same way as on a
single processor system.
John
johnDOTmorrisonATtescoDOTnet
--
"Beware of bugs in the above code; I have only proved it correct, not tried
it." - Donald Knuth
|
|
| | From: | Giancarlo Niccolai | | Subject: | Re: Synchronization Vs Concurrency | | Date: | Sun, 19 Dec 2004 12:22:12 +0100 |
|
|
| John Morrison wrote:
> Jim Rogers wrote:
>> "Zahid Faizal" wrote in
>> news:1100639531.519829.256120@c13g2000cwb.googlegroups.com:
>>
>>> I have often heard about synchronization and concurrency being talked
>>> in tandem. I kinda know what this is all about ----- semaphores,
>>> controlled access to critical resources blah blah blah. However, I
>>> am not sure of the delineation between the two. What is the
>>> "synchronization" part in this, and what is the "concurrency" part?
>>> Can the two be talked about in isolation, or do they always exist in
>>> tandem?
>>>
>>> Any insight will be appreciated.
>>
>> (I have trimmed the massive cross postings)
>>
>> Synchronization is one tool used in concurrent programs to ensure
>> data integrity for data shared by several concurrent tasks.
>>
>> Concurrent programs can be written without any form of
>> synchronization. In that case, either the concurrent tasks share
>> no data, or any data corruption due to unsynchronized sharing
>> is either impossible or unimportant.
>>
>> Concurrency deals with the creation, scheduling, and lifetime
>> of concurrent tasks. Synchronization deals with the protected
>> sharing of data and state information between concurrent tasks.
>>
>> A single-threaded program is completely synchronous.
>> The purpose behind multi-threaded applications is to achieve
>> greater asynchronous behavior, allowing many things to be
>> done at the same time. Many multi-threaded programs still need
>> some degree of synchronization to facilitate reliable communcation
>> between concurrent tasks.
It would be interesting if this guy can provide any example of real-world
program not needing synchronization in any point. As all the interesting
system I/O functions (i.e. read, write and so on) causes at least implicit
synchronization on the used resources (if not on the same file, at least on
the same physical device), those concurrent non synchronized programs may
only be pure computations without input and output, for which I can't see
many uses.
(Synchronization does not mean lock. Also lock free lists are a mean of
synch).
Also, I do not completely agree with the previous sentence: the lifetime of
threads is an important synchronization fact in real world programs. In
example, there MAY be a set of pure calculation threads, without I/O, which
then dies and handle back their data to a single thread storing it
somewhere via I/O. But THAT thread will then have to synchronize waiting
for the lifetime of all the interested threads to be elapsed.
>>
>> Asynchronous communication often takes the form of signals.
>> All the operating systems I am familiar with provide some set of
>> signals to indicate a desired change of state in the concurrent
>> task receiving the signal. Other than the signal itself, no data
>> is sent from one concurrent task to the other in the communication
>> of a signal. This is the reason that signals are very limited in
>> their applicability as a communication between concurrent tasks.
>> Most operating systems only support signals at the process level,
>> and not at the thread level. Linux is an exception because of the
>> strong similarity between threads and processes.
>>
From how this is said, it seems that the author consider signal-to-threads
an interesting feature. Linux is an exception because of a design flaw (and
not because of a desired feature), that is anyhow HIDDEN as far as possible
by the pthread_ libraries for linux (and by the system where it is capable
to do so), and that cannot be relied on. So, linux won't support signal at
thread level forever, and AFAIK relying on signals to threads in linux may
well cause broken programs between kernel/pthread releases.
Bests,
Giancarlo Niccolai.
|
|
| | From: | Jim Rogers | | Subject: | Re: Synchronization Vs Concurrency | | Date: | Mon, 20 Dec 2004 03:51:32 GMT |
|
|
| Following is a conversation initiated when I received a
direct email from John Morrison:
>Jim Rogers wrote:
>> Hello John,
>>
>> I have no significant disagreement with what you say below.
>> I do have some remarks.
>
> I'm glad about both those statements!
>
>>
>>> The only other thing which needs emphasis here is real *vs*
>>> apparent concurrency. The former is available (indeed, inevitable!)
>>> on systems with more than one processor, and needs some "tricks"
>>> (/e.g./ spin locks) at the system level for guarding OS resources
>>> against simultaneous access by OS code on different processors. With
>>> one processor only, all concurrency is apparent, since one processor
>>> can - obviously! - do only one thing at a time.
>>
>> While it is true that a single processor is traditionally capable of
>> doing only
>> one thing at a time, our view of a single processor may be changing
>> in the very near future.
>>
>> AMD has already demonstrated dual-core 64-bit CPU chips.
>> Intel attempts to parallelize instructions through their
>> Hyperthreading technology.
>
> Of course,various ,manufacturers have been trying to (but not
>necessarily succeeding in) doing that for years! Intel had some _very_
>subtle bugs with its chips early on ... in 32-bit days, which AMD -
>fortunately! - managed not to emulate.
>
>>
>> Even on traditional multi-processor systems your definition of "true"
>> concurrency only holds true when the number of threads does not exceed
>> the number of processors. As soon as the number of threads exceeds the
>> number of processors you are faced with some level of "true"
>> concurrency mixed with some level of apparent concurrency. A
>> single-processor system resides at one end of this mixture. A
>> massively parallel system resides near the other end of this mixture.
>
> Indeed: although I note that, should I grant you this distinction in
>_precisely_ your terms, there is a(n) (inevitable) blurring between the
>concept of "thread" and that of "process". Such a blurring is
>inevitable,
>however. I started off working in single-threaded environments (or to
>put it
>another way, thread and process were identical: each process had one
>thread - end of story). Later on, processes had the *possibility* - no
>more - of being constituted as more than a single thread. This meant
>that
>better synchronization/mutual exclusion mechanisms had to be provided
> ...
>
>>
>>>
>>> At the process/thread level, there should be no difference: even
>>> if two (or more) threads of the same process are running
>>> simultaneously on different processors, the OS's mutual exclusion
>>> provisions - whatever names they have on the OS you are using - are
>>> used in exactly the same way as on a single processor system.
>>>
>>
>> Absolutely true.
>>
>> Proper design of concurrent programs should be portable across
>> hardware configurations. The most apparent difference may be in
>> performance.
>> This is one of the reasons I really like using a high level language
>> that supports
>> concurrency primitives directly in its syntax. The portability of the
>> program can
>> then extend beyond hardware changes to OS changes through a simple
>> re-compilation of the source code.
>>
>> I have used Java and I have used Ada for these purposes. I prefer
>> Ada's concurrency features over Java's concurrency features. I find
>> Java's concurrency and synchronization features to be poorly designed
>> while I
>> find Ada's concurrency features to be simple to use, efficient, and
>> robust.
>>
> When I first met (and hence used) _true_ multiprocessor systems,
> they
>had progressed from ASMP (one master processor) to SMP, and this made a
> very
>great difference to how one coded such things as drivers, which behave
> as
>actual parts of the OS. Specifically, if my memory as good as any
> computer's
>ought to be (very unlikely, I admit), it was between V4.4 and V4.5 of
>VAX/VMS when specific macros were introduced for driver writers to
>ensure
>that "unsupportable" page faults did not occur on multiprocessor
>systems.
>(There were "tricks" used by driver writers before that which could
>_only_
>be guaranteed to work on single processor systems, or on older ASMP
>systems.
>Fortunately, if your original code was properly written, the changes
>required were minimal.) I note that my experience (at the lowest level)
>and
>yours (in high level languages) are utterly different.
>
> I have never used either Ada or Java "in anger", so to speak.
>However, I
>"know both languages in theory": and can *well* understand why you would
>prefer the Ada /rendezvous/ to anything which Java has to offer.
The Ada /rendezvous/ was introduced into the language during the 1983
standard. The /rendezvous/ mechanism is useful in some situations, but
tends to be a heavy-weight approach to data synchronization such as a
shared buffer. The 1983 version of the /rendezvous/ also exhibited a
nasty tendency for deadlocks and priority inversion.
Both those issues were handled in the 1995 Ada standard. Ada95 fixed
the deadlock and priority inversion issues with the /rendezvous/.
It also introduced an efficient and light-weight mechanism for data
sharing between tasks. This also allowed tasks to execute in a much
more asynchronous manner. The new mechanism was named /protected
objects/. A protected object maintains its own private data. There
are three kinds of methods which one may create for a /protected
object/. Protected functions provide read-only access to the data
or state of a /protected object/. Protected procedures provide
unconditional read/write access to the data and state of a
/protected object/. Protected entries provide conditional
read/write access to the data and state of a /protected object/.
Protected procedures and protected entries are granted exclusive
access to a /protected object/. Protected entries also automatically
maintain a queue of tasks waiting for the entry boundary condition
to evaluate to TRUE. Protected functions, since they do not alter
the state of the /protected object/, are allowed shared access.
/Protected objects/ are passive units in concurrency. Any call to
a protected object's method(s) are executed in the calling task.
Ada tasks are active concurrency units.
The following code demonstrates a protected object using all three
kinds of protected methods.
As with most other Ada code, protected objects are required to have
separate specifications and implementations.
protected Counting_Semaphore is
procedure Release;
entry Acquire;
function Count return Natural;
private
Lock_Count : Natual := 0;
end Counting_Semaphore;
protected body Counting_Semaphore is
procedure Release is
begin
if Lock_Count > 0 then
Lock_Count := Lock_Count - 1;
end if;
end Release;
entry Acquire when Lock_Count < 5 is
begin
Lock_Count := Lock_Count + 1;
end Acquire;
function Count return Natural is
begin
return Lock_Count;
end Count;
end Counting_Semaphore;
This example provides a counting semaphore allowing up to
5 tasks to hold the semaphore. Any task calling the Acquire
entry when Lock_Count < 5 will immediately acquire the
lock. If the Lock_Count is not < 5 the calling task will
suspend and be added to the entry queue maintained for
Acquire. When the procedure Release is called the entry
boundary condition for Acquire is automatically evaluated.
If the condition evaluates to TRUE and a task is in the
entry queue, the waiting task is allowed to procede and
the body of acquire is executed. This evaluation and
execution is actually performed by the task calling the
Release entry, avoiding unnecessary context switching.
My experience with Java reveals nothing even remotely similar
to these capabilities.
>
> Last, but not least: I seem now to be replying to you personally,
>not to
>the group. Just how this happened, I am unsure: but, to ensure proper
>non-exclusion, Jim: if you think that this reply is worthy of it, please
>post it to the group with any further comments, and an explanation.
>(Probably easier that way than if _I_ do it.)
Jim Rogers
|
|
|
