How was multiprogramming brought into computer languages

Question

I am going through the book operating system principles by Hansen . Hansen begins the third chapter on concurrent programming by discussing interleaving of the machine instructions of processes being run at the same time .

Then it begins with providing the language notation for multiprogramming and writes the following program whose purpose is to copy records from one sequence to another .

procedure copy (var f, g: sequence of T);
var s, t: T; 
 completed: boolean; 
begin 
 if not empty(f) 
 then 
 begin 
 completed:= false;
 get(s, f);
 repeat 
 t:= s; 
 cobegin 
 put(t, g); 
 if empty(f) 
 then 
 completed:= true 
 else 
 get(s, f); 
 coend 
 until completed; 
 end
 end 

(If possible please tell me the name of the language ,was it Algol60 ?Anyways let's call it X )

All fine till here , a keyword or statement was introduced into the language X , which would perform multiprogramming. But I would like to know how was the language changed so as to incorporate this new keyword or function for multiprogramming . What changes were done to language X,owing to which X which was capable of supporting one execution at a time could now support more than one at a time ?

P.S : Hansen described about a single -user operating system called "Solo ".There he somehow means Pascal supports development of concurrent programs. The answer to the question in this post can be approximated by an answer to "what were the differences between languages which supported concurrent programs and those who didn't "?

I could have written "what are the differences" in the auxiliary question but I wanted to know how languages started having constructs for concurrent programming ,that's why I wrote "What were the differences".

• rcgldr: It's my impression that the first high level language to get system specific extensions for parallel operations was Fortran. For example the CDC 6600 (1960's) followed by other mainframes and/or super computers that supported "vector" operations. These early extensions were system specific and non-portable. Some of the early code was also done with assembly. Multi-processing for user programs is mostly handled via function calls related to synchronization and memory sharing.
• itp dusra: @rcgldr : That's what I want to know , The process of adding Function calls related to synchronization and memory sharing to the language . Could you elaborate a bit more ?
• rcgldr: For the function calls, no language changes are needed. For example, in the case of windows, the functions are CreateThread(), CreateProcess(), CreateSemaphore(), WaitForSingleObject(), WaitForMultipleObjects(), CreateFileMapping (using invalid file handle) for shared memory between processes, and other functions. The libraries would need to support multi-threading, but the actual language needs no changes.
• itp dusra: @rcgldr : were these functions included in the language ?I mean as in keywords , were these functions keywords in the languages of those time ? How were the libraries created ?
• itp dusra: @rcgldr : But Posix didn't exist back then . What are these libraries of visual studio written in ?

Answer 1

If you're really interested in all that's needed, your best choice would be to look at the standardisation process of C++11 by the ISO Working Group on C++ (WG21). All the relevant papers are available online.

Keywords are not important at all. In fact, C++ defined just one new keyword (thread_local) and that's mostly for convenience.

What is important is the memory model. Programs work by altering state, which logically exists in memory. With multi-programming, there are concurrent changes to the state of a program. This requires a reasonably precise definition of said state. To make things more complex, you want a definition of state that is compatible not just with the abstract language definition, but which is also implementable on real hardware. That gets you into problems like registers and caches (state outside ordinary memory), the exact details of which vary significantly between real hardware architectures.

A key insight here is the sequenced concept. Two C++ operations can be sequenced, which means they cannot operate at the same time. This means they are allowed to use the same data in memory without risk. A special kind of sequencing is sequenced before, which means one operations happens before the other. This means a write sequenced before a read guarantees that the read sees the value of the write.

With C++11, there are library calls defined which can guarantee such sequencing across different threads of execution.

The comments of @rcgldr suggest that function calls like WaitForSingleObject alone are sufficient, but this isn't actually the case. You need an associated memory model. That's why you have Win32 macro's like InterlockedIncrement - these macro's provide the mapping between the language memory model and the hardware memory model. There's no Read(DWORD*) macro, though - the language memory model is that aligned DWORD reads are atomic, and it's implicitly assumed that the compiler can efficiently map every read to atomic hardware instructions.

• itp dusra: My gratefulness for your reply .By keywords I meant words pertaining to starting simultaneously running programs . Do parallelly or simultaneously programs actually fall under sequenced programs ?I am a bit confused as cooperating programs are supposed to operate together and sequential ones don't.
• itp dusra: isocpp.org/forums...were you referring to this link . Could you tell me where in this link should I look for ?
• itp dusra: :What exactly the library calls written in ?