Describe some of the problems as seen by Dr Terman with MIMD

Solution

CACHE with MIMD:In multiprocessor machines where several processors require a copy of the same memory block, the maintenance of consistency among these copies raises the so-called cache coherence problem which has three causes:

From the point of view of cache coherence, data structures can be divided into three classes:

There are several techniques to maintain cache coherence for the critical case, that is, shared writable data structures. The applied methods can be divided into two classes:

SHARED MEMORY with MIMD: The main problem one is confronted with in shared-memory systems is that of the connection of the CPUs to each other and to the memory. As more CPUs are added, the collective bandwidth to the memory ideally should increase linearly with the number of processors, while each processor should preferably communicate directly with all others without the much slower alternative of having to use the memory in an intermediate stage. Unfortunately, full interconnection is quite costly, growing with O(n²) while increasing the number of processors with O(n).

Shared memory model scalability beyond thirty-two processors is difficult, and the shared memory model is less flexible than the distributed memory model.

Shared memory can be UMA or NUMA. Problems in both types are as follow:

Uniform memory access (UMA)

Non-uniform memory access (NUMA)