rw-book-cover

Metadata

Highlights

  • Normies: Problem solvers with antipathy for black boxes, gratuitous complexity and external dependencies At Normconf, “normie” participants often embodied the three fundamental psychological needs identified in Self-Determination Theory: autonomy, competence, and relatedness. They talked about how they autonomously solved all kinds of problems, related on the pains and gains of their roles, and showed off their competence across the board, in solving problems. What they did, was what I also did as a data engineer: We start from a business problem, and work back through what needs to be done to understand and solve it. By very definition, Normie is someone not very specialised at one thing or another, and in our field, even data engineers are jacks of all trades. What undermines the Normie mission are things that clash with the basic needs, from uncustomisable products, to vendors that add bottlenecks and unreliable dependencies. (View Highlight)
  • I saw an acute friction between data engineers and Python-first analysts, mostly around the fact that data engineers easily become a bottleneck and data scientists are forced to pick up the slack. And of course, this causes other issues that might further complicate the life of the data engineer, while still not being a good solution for the data consumers. (View Highlight)
  • We finally launched dlt mid 2023 to the general public. Our initial community was mostly data engineers who had been using dlt without docs, managing from reading code. As we hoped a lot of “normies” are using dlt, too! (View Highlight)
  • In the early days of dlt we talked to Sebastian Ramirez from FastAPI who told us that he spends 2/3 of his FastAPI time writing documentation. In this vein, from the beginning docs were very important to us and we quickly adopted our own docs standard. However, when we originally launched dlt, we found that different user types, especially Normies, expect different things from our docs, and because we asked for feedback, they told us. (View Highlight)
  • The docs helper has been running for a year and we currently see around 300 questions per day. Comparing this to other communities that do AI support on Slack, that’s almost 2 orders of magnitude difference in question volume by community size. (View Highlight)