[{"content":"Infrastructure as Code (IaC) is an approach to managing and provisioning computer systems in which infrastructure is treated as software. This means that instead of manually configuring and setting up servers, storage, and networks, data engineers can use code to define and deploy their infrastructure.\nIn the past, data engineers had to manually set up and configure their infrastructure, which was time-consuming and error-prone. With IaC, they can use code to automate the process, making it faster, more efficient, and more reliable.\nThere are many benefits to using IaC in the data engineering field. Some of the main benefits include:\nReproducibility: With IaC, data engineers can easily reproduce their infrastructure in different environments. This is useful for testing, staging, and production environments, as well as for disaster recovery. Collaboration: IaC allows data engineers to collaborate more effectively on infrastructure projects. They can use version control systems to track changes to their code, and they can use code review tools to ensure that their code is of high quality. Documentation: Code is a form of documentation, and IaC provides a clear and concise way to document the data engineering infrastructure. This is useful for onboarding new team members and for maintaining the infrastructure over time. Scalability: IaC makes it easy for data engineers to scale their infrastructure up or down as needed. They can use code to define their infrastructure in a way that is flexible and adaptable to changing needs. There are many tools and technologies that data engineers can use to implement IaC. Some popular options include:\nTerraform: Terraform is an open-source IaC tool that allows data engineers to define and manage their infrastructure as code. It supports a wide range of infrastructure providers, including AWS, Azure, and Google Cloud.\nPuppet: Puppet is an open-source IaC tool that allows data engineers to automate the configuration of their infrastructure. It uses a declarative language called Puppet DSL to define the desired state of the infrastructure, and it ensures that the infrastructure is always in that state.\nAnsible: Ansible is an open-source IaC tool that allows data engineers to automate the deployment and configuration of their infrastructure. It uses a simple YAML syntax to define the desired state of the infrastructure, and it can be used to manage a wide range of infrastructure types.\nIn conclusion, IaC is an important approach for data engineers who want to manage and provision their infrastructure in a more efficient and reliable way. By using code to define and deploy their infrastructure, they can enjoy benefits such as reproducibility, collaboration, documentation, and scalability. There are many tools and technologies available to help data engineers implement IaC, and it is worth considering which one is the best fit for their needs.\nIn my next post I will be going through 3 of the most popular IaC tools, their use-cases and their trade-offs.\n","date":"20 May 2023","externalUrl":null,"permalink":"/posts/infrastructure_as_code/","section":"","summary":"","title":"Infrastructure as code In Data Engineering","type":"posts"},{"content":"","date":"9 February 2026","externalUrl":null,"permalink":"/posts/","section":"","summary":"","title":"","type":"posts"},{"content":"","date":"9 February 2026","externalUrl":null,"permalink":"/tags/adr/","section":"Tags","summary":"","title":"Adr","type":"tags"},{"content":"","date":"9 February 2026","externalUrl":null,"permalink":"/tags/architecture/","section":"Tags","summary":"","title":"Architecture","type":"tags"},{"content":"","date":"9 February 2026","externalUrl":null,"permalink":"/","section":"Caleb Ejakait","summary":"","title":"Caleb Ejakait","type":"page"},{"content":"","date":"9 February 2026","externalUrl":null,"permalink":"/tags/data/","section":"Tags","summary":"","title":"Data","type":"tags"},{"content":"","date":"9 February 2026","externalUrl":null,"permalink":"/tags/software/","section":"Tags","summary":"","title":"Software","type":"tags"},{"content":"","date":"9 February 2026","externalUrl":null,"permalink":"/tags/","section":"Tags","summary":"","title":"Tags","type":"tags"},{"content":"The ultimate output of knowledge work,in this case software architecture and engineering, is decision making. Regardless of what form it takes, recording the thought process of key decisions is an important part of maintaining clarity before, during and after decision are agreed upon, to help prevent hind-sight bias and to allow your future self and others, understand the environment in which the decisions were made.\nIn this post I will be sharing what I have been learning about Architecture Decision Records their applications in software and data projects.\nWhat are ARDs anyway? # Architecture Decision Records are essentially a paper trail for significant architectural decisions that you make in the lifecyle of a project. I will mostly refer to them as ADRs from now on.\nIn my years as an software engineer and data engineer, I have scarcely, if at all come across this ADRs. Documentation has taken many forms in the projects I have been apart of; abandoned Notion pages, diagrams and sparsely detailed text files about what was going to be implemented or has already been implemented, or no documentation at all. When decison change, ADRs help track this by introducing states to these documents i.e Propose, Accepted, Rejected, Superseded\nWhy should you care? # Architectural decisions are some of the most costly in any organization. They have impacts on how easy it will be to add features or how resilient and flexible the system will be to changing requirements of the business. Interrogating and justifying these decisions with a decent amount of rigour, within the required time-frame becomes that much more important. Not to mention the framework can be adopting to many other fields and types of decision processes.\nHow do you know what to document? # Not every decision warrants a paper trail; the administrative overhead would be a nightmare. Michael Nygard describes \u0026ldquo;architecturally significant\u0026rdquo; decisions as those affecting:\nStructure and construction techniques. Non-functional characteristics, for example how it affects scalability or security. Dependencies and interfaces Ask yourself: Is this decision reversible, and what is the cost of reversing it?\nOlaf Zimmermann goes into great detail on this in his blog post about what criteria to use. regarding decision criteria\nLightweight ADRs: Setting Them Up # I\u0026rsquo;ll be the first to hold up a cross to administrative overhead and because gaining initial velocity on the habit of documentation is hard enough, lightweight ADRs make for the best format to help reap the benefits of documenting your decision without dramatically affecting your workflow.There are even tools that help you manage ADRs, which you can find in this gihub repo to help grease the wheels.\nIn my projects, I simply add an /adr folder consisting of one Markdown document per decision:\nADR 001 - Postgres for Operational Database.md\nADR 002 - Microservice Architecture for Orders Module.md\nI include diagrams directly in the folder and commit everything to source control. This allows the documentation to version-match the code. When a decision changes, I don’t edit the old ADR; I create a new one, mark the old one as Superseded, and link to the replacement.\nThe lifecycle of ADRs # The lifecycle of these documents vary and are affected by team dynamics and roles and responsibilities of different stakeholder in the decision making process. There is always a main contact person or team responsible for communicating, publishing and maintaining these documents where maintenance is moving an adr from one state to the next.\nI have come to interprete the states as below:\nProposed: The draft phase; getting stakeholders to poke holes in the logic. Accepted: The green light; ensuring everyone is aligned on the path. Rejected: Recording why we said \u0026ldquo;no\u0026rdquo; so we don\u0026rsquo;t repeat the debate next year. Superseded: Signaling that a new ADR has taken its place, ensuring there is only one \u0026ldquo;source of truth.\u0026rdquo; Conclusions # Documentation is a habit and one that is important especially with the changing landscape of tools and technologies that teams adopt. Though many of the organizational consideration that go into documentation frameworks haven\u0026rsquo;t been touched on here, the rigour required to fill-in and socializing ADRs alone should help stave off some of the pitfalls that come with insufficient documentation and decision silos that are created as a result. Where ADRs live is just as important as writing them and it affects how they are versioned and maintained which is, in my opinion, the biggest battle with technical documentiation.\nResources # More on ARDs and Templates. References # Communicating and documenting architectural decisions - David Ayers Documenting Architecture Decisions - Michael Nygard 2011 ","date":"9 February 2026","externalUrl":null,"permalink":"/posts/ards_1/","section":"","summary":"","title":"The Power of the Paper Trail: Architecture Decision Records (ADRs)","type":"posts"},{"content":"","date":"20 May 2023","externalUrl":null,"permalink":"/tags/data-engineering/","section":"Tags","summary":"","title":"Data Engineering","type":"tags"},{"content":"","date":"20 May 2023","externalUrl":null,"permalink":"/tags/iac/","section":"Tags","summary":"","title":"Iac","type":"tags"},{"content":"","date":"20 May 2023","externalUrl":null,"permalink":"/tags/terraform/","section":"Tags","summary":"","title":"Terraform","type":"tags"},{"content":"","externalUrl":null,"permalink":"/pages/","section":"","summary":"","title":"","type":"pages"},{"content":"","externalUrl":null,"permalink":"/pages/about/","section":"","summary":"","title":"","type":"pages"},{"content":"","externalUrl":null,"permalink":"/authors/","section":"Authors","summary":"","title":"Authors","type":"authors"},{"content":"","externalUrl":null,"permalink":"/categories/","section":"Categories","summary":"","title":"Categories","type":"categories"},{"content":"","externalUrl":null,"permalink":"/series/","section":"Series","summary":"","title":"Series","type":"series"}]