Software Engineering as Social Science

A codebase does not assemble itself. Every module boundary, every interface, every awkward integration layer is the fossil record of a decision made by a group of people, under a deadline, with a particular reporting structure and a particular set of things they were and weren't allowed to change. Read a large system's architecture correctly and you are reading an org chart; read its commit history correctly and you are reading minutes from meetings that were never written down. This lecture treats the codebase as a social artifact, and asks what a social scientist would ask of it: who talked to whom, who had power over what, and what behaviour did the system's own design incentivise once it existed.

Conway's Law in Action

In 1968, Melvin Conway made an observation that has proven far more durable than the memo it appeared in: "organizations which design systems ... are constrained to produce designs which are copies of the communication structures of these organizations" [1]. The claim is not metaphorical. If two teams do not talk to each other regularly, the interface between the components they own will accumulate friction and drift, because neither team has a cheap way to renegotiate it. If a team is organised into three sub-teams, the system it builds tends toward three major components, because the module boundary is the path of least social resistance, not necessarily the path of least technical resistance.

This has since been tested rather than merely observed. MacCormack, Rusnak and Baldwin's empirical study of comparable commercial and open-source products found measurably tighter internal coupling in the code built by tightly-coordinated commercial teams than in the more loosely-coordinated open-source equivalents — the "mirroring hypothesis" held up as a testable, falsifiable claim about software architecture, not just a folk saying [2]. Herbsleb and Grinter's field study at a telecommunications company went further: they watched a single, previously coherent module get split across two newly-separated teams and documented the coordination failures — stale documentation standing in for conversations that used to happen face-to-face — that followed directly from the org change, months before the resulting defects showed up in the code [3]. Nagappan, Murphy and Basili later showed the effect statistically at Windows Vista's scale: organisational metrics (how many people touched a component, how distributed they were) predicted failure-proneness better than any code complexity metric measured on the same components [4].

graph TD subgraph "Team structure" T1[Team A] -.rarely talks to.- T2[Team B] end subgraph "System architecture" M1[Module A] ---|thin, brittle interface| M2[Module B] end T1 --> M1 T2 --> M2

The practical reading of Conway's Law is not fatalistic. It is diagnostic: an unexpectedly painful interface in the code is a reliable signal to go and look at the team boundary sitting behind it, rather than only at the code itself.

Incentive Alignment & System Behaviour

Software doesn't just reflect the social structure that built it — once deployed, it reshapes the behaviour of the people who use it, because interfaces are incentive structures. A code review tool that surfaces "lines changed" prominently will, over time, produce reviewers who scrutinise large diffs more heavily than small ones regardless of actual risk, because the tool has made size the salient signal. A support ticketing system that rewards fast closure over correct resolution will train agents to close tickets fast. This is Goodhart's territory — "when a measure becomes a target, it ceases to be a good measure" — and it applies as much to the metrics a system exposes to its own users as to the metrics management uses to judge a team.

Curtis, Krasner and Iscoe's landmark field study of large-system design processes documented this at the level of entire projects: the biggest determinant of whether a design decision would stick was not its technical merit but whether the organisational structure gave the people who understood the constraint enough standing to enforce it against schedule pressure [5]. The lesson generalises down to interface design: every constraint a system enforces, and every constraint it merely suggests, is shaping what "doing the job well" comes to mean for the humans using it — often faster than any training programme could.

Software Maintenance as Social History

A mature repository's commit log and pull-request history is a primary source, in the historian's sense: contemporaneous, produced by the participants themselves, and usually more honest than any after-the-fact retrospective, because nobody was writing it for posterity. Reading it as social history rather than as a changelog reveals things a changelog is designed to hide — the three abandoned branches attempting a rewrite that never landed, the terse commit message on a Friday afternoon that turns out to be the exact moment two subsystems stopped being allowed to talk directly, the review thread where a junior engineer's objection was overruled and the bug that objection predicted six months later.

This is now a recognised empirical technique, not just a reading exercise. Mockus, Fielding and Herbsleb's comparative study of Apache and Mozilla mined contribution logs to reconstruct each project's actual (as opposed to nominal) core-and-periphery social structure, and found the two open-source projects organised their informal hierarchies quite differently despite superficially similar processes [6]. Bird and colleagues went further, building socio-technical networks directly from commit co-authorship and modification-of-the-same-file data to predict which components would fail — the social graph, not the dependency graph alone, carried real predictive signal about where defects would cluster [7].

Practical Exercise: Mapping the Informal Network. Choose a repository with real history (ideally 100+ contributors or 2+ years of activity). Build a contributor graph from commit co-modification of the same files, and a communication graph from pull-request review comments. Compare the two. Identify: (a) any knowledge silo — a component only one or two people ever touch — and what happens to review quality on it; (b) any place the two graphs disagree, where people who never review each other's code are nonetheless modifying the same files. What does that disagreement predict about the module boundary sitting between them?

References

  1. Conway, M. E. (1968). How do committees invent? Datamation, 14(4), 28–31.
  2. MacCormack, A., Rusnak, J., & Baldwin, C. Y. (2012). Exploring the duality between product and organizational architectures: A test of the mirroring hypothesis. Research Policy, 41(8), 1309–1324. https://doi.org/10.1016/j.respol.2012.04.011
  3. Herbsleb, J. D., & Grinter, R. E. (1999). Splitting the organization and integrating the code: Conway's law revisited. Proceedings of the 21st International Conference on Software Engineering (ICSE '99), 85–95. https://doi.org/10.1145/302405.302455
  4. Nagappan, N., Murphy, B., & Basili, V. (2008). The influence of organizational structure on software quality: An empirical case study. Proceedings of the 30th International Conference on Software Engineering (ICSE '08), 521–530. https://doi.org/10.1145/1368088.1368160
  5. Curtis, B., Krasner, H., & Iscoe, N. (1988). A field study of the software design process for large systems. Communications of the ACM, 31(11), 1268–1287. https://doi.org/10.1145/50087.50089
  6. Mockus, A., Fielding, R. T., & Herbsleb, J. D. (2002). Two case studies of open source software development: Apache and Mozilla. ACM Transactions on Software Engineering and Methodology, 11(3), 309–346. https://doi.org/10.1145/567793.567795
  7. Bird, C., Nagappan, N., Gall, H., Murphy, B., & Devanbu, P. (2009). Putting it all together: Using socio-technical networks to predict failures. Proceedings of the 20th International Symposium on Software Reliability Engineering (ISSRE 2009), 109–119. https://doi.org/10.1109/ISSRE.2009.17