Big PL Ideas

APL and some other array-oriented languages get arrays right - they are fundamental to the language and using them is easy and direct. This ease is reflected in dedicated syntax for manipulating arrays in multiple ways. As mentioned recently on The Array Cast, most of the array languages have implicit mapping (1 + 2 3 4 is 3 4 5), but some fundamental operations in lisps have implicit reduction, which is why (+ 1 2 3) evaluates to 6. In my opinion, this also counts toward "getting arrays right", but I'm also biased in favor of APL-style notation.

The biggest surprise to me is that neither implicit mapping nor implicit reduction is mainstream yet. Perhaps the closest mainstream analog to these ideas is NumPy's broadcasting behavior.

Some mainstream languages actually do build in concurrency to the language and its run-time environment. JavaScript's approach of having a global event loop to which you can register event handlers certainly counts. Go's CSP channels handle concurrency differently, and Erlang or Elixir's actor model are yet another approach. But all of these examples build something into the language at a fundamental level, rather than forcing programmers to find and choose libraries to implement these features.

Unfortunately, most array languages focus solely on SIMD parallelism (one kind of concurrency) and ignore other kinds of concurrency. Really my biggest complaint about array languages is that they are overtly batch-oriented, despite most of them being interactive and interpreted languages. Asynchronous event handling is awkward at best in array languages.

Errors happen and specifications change over time. Even a program which was once proven to be correct can eventually become incorrect. The ability to modify parts of a long-running system without disturbing other parts is how I define resilient. Similar to how your skin can heal a cut on your hand without requiring a full arm replacement, a resilient software system can heal one sub-part without replacing the larger part.

The next software system I hope to learn will be Elixir or Erlang, because of the facilities for resilience provided by the virtual machine they both use (BEAM). From the pure programming language side of things, Erlang and Elixir aren't that interesting to me. They're pure/functional and use pattern matching and recursion rather than state and loops, which is fine. My brain prefers mapping and reduction over either looping or recursion, but I know that deep down they're all gotos, so whatever the top layer of syntax looks like, I am confident I will be able to use it to express my intent. I expect the languages make some things easier but others harder relative to more mainstream imperative/procedural style languages. But I'm comfortable enough with pure functional languages that I don't expect this to be a terrible learning curve.

Here I'm referring to parallelism in the most general sense - redundant computing elements operating concurrently in a distributed system to solve parts of the same problem or even completely separate problems.

I'm currently reading Alexey Radul's Ph.D thesis about propagation which seems to be where we're all heading eventually. In this design, stateful cells hold information and stateless propagators convey information among cells. In a hardware system this could be realized by a processor-in-memory architecture, with RAM providing the cell with multiple nearby CPUs providing propagation in multiple directions. Systolic arrays (like those used in Tensor Processing Units) are not exactly related, but achieve a similar result. Chuck Moore's GreenArrays company offers a "multi-computer chip" with no global clock in which independent computers act on input/output pins and communicate asynchronously with their immediate neighbors. This seems like a hybrid between processor-in-memory and systolic array architectures.

A key part of Radul's thesis is that cells accumulate information rather than store results directly - why this is important isn't really clear to me yet, but I'll keep reading. One thing I have come to see as fundamentally true is that all systems are gradually evolving into distributed systems. Even the chips you can buy from Intel which are ostensibly "a CPU" are not one chip or even one type of chip, but many different types of chips sharing a common substrate and networked together to perform the more general task of being CPU-like.