1

 Coded Computation I
 John E Savage

2

 Coded Computation: Replace the repetition code with something more
efficient.
 First we review work from the 50s and 60s on coded computation,
highlighting some negative results.
 Next we look at Dan Spielman’s paper Highly FaultTolerant Parallel
Computation Procs 37th Annl IEEE Conf. Foundations of Computer Science,
pp. 154163, 1996.
 Spielman’s approach realizes reliable circuits with unreliable gates
more efficiently than the “von Neumann” method.

3

 Goal: Reliably compute function g_{k}(x_{1}, x_{2})
= y where x_{1}, x_{2} and y are of length k.
 Method:
 Encode x_{1}, x_{2}. Compute G_{n}(E(x_{1}),E(x_{2}))
= E(y). x_{1}, x_{2 }and y are encoded in the same code
C (prevents cheating).
 Result of computation is z, a noisy version of y.
 Decode z to a codeword in C.
 The challenge is to choose codes to ensure that z can be decoded
correctly in the presence of errors.

4

 We explore the idea that G_{n}, like g_{k}, is local, i.e.
operates componentwise on E(x_{1}), E(x_{2}).
 [Winograd 62] has shown that when G_{n} is local and input and
output codes are the same, n ≥ 2e+1 to cope with e errors (See
following.) Can’t beat repetition!
 Here the amount of redundancy required to cope with faults is n/k. As
with communication, the “rate” is k/n.

5

 Let y = g_{k}(x_{1}, x_{2}) = AND(x_{1},
x_{2}). Assume G_{n} = AND.
 E(y) = AND(E(x_{1}), E(x_{2})) = E(AND(x_{1}, x_{2}))
 Note: a Þ b if and only if
AND(a,b) = a.
 If x_{1} Þ x_{2}
(x_{1i }Þ x_{2i}
for each i), E(x_{1}) Þ E(x_{2}). Follows ‘cause AND(x_{1}, x_{2})
= x_{1} implies AND(E(x_{1}), E(x_{2})) =E(x_{1})
 Since g_{k}(0^{k}, x_{2}) =0^{k}, E(0^{k})
= AND(E(0^{k}), E(x_{2})) must be 0^{n} because
0^{n} implies all E(x). Similarly E(1^{k}) = 1^{n}.

6

 If x_{1} is all 1s, E(y) = E(x_{2}). If we set x_{2}
to all 0s, then set k bits to 1 one at a time, some bits in E(x_{2})
increase from 0 to 1. Each change in x_{2} generates a new
codeword.
 To tolerate e errors, at least (2e + 1) 1s must be added at each step.
Thus, n ≥ (2e + 1)k.
 Same for OR, n ≥ (2e+1)k. ¨

7

 Coded computation cannot be local without paying a high price.
 Must consider nonlocal coded computation.

8

 ddim hypercube executing normal algorithms
 Normal algorithms exchange info between processors along the same
dimension on a step.
 Other parallel models can be mapped to hypercube with polylog time
loss.
 Each hypercube node is a processor
 Processors exchange info with neighbors and compute in synchronism.
 Each processor has own instruction stream.

9

 Encodes tuple of processor states (σ_{j} ) using a RS code p(x).
 Encodes tuple (w_{j,t} ) of instructions on tth step for
processors using a RS code q(x).

10

 The function F(σ_{j},
β, w_{j,t}) combining processor state σ_{j}
for the jth processor with neighbor state σ_{s} and
instruction word w_{j,t} on tth step is implemented using
interpolation polynomial f(u,v,w).
 States encoded with polyn. p(x), permuted states with p’(x), and
instructions with q(x).
 Output is f(p(x),p’(x),q(x)) evaluated
at x'F Output is word
in different RS code!

11

 A 1D interpolation polynomial
 A 2D interpolation polynomial
 f(σ_{j}, β, w_{j,t})
represented by
