The First GitOps Policy Engine
Bringing back visibility and control to codebase management in the DevOps era
The VP engineering of a well known international software company led one of our recent Brainfest sessions. We listened intently as he told the story of his engineering group over the last five years.
“Like most software companies we had one big monolithic architecture for our applications and a predetermined technology stack. But the market conditions were changing rapidly around us. After years of dominating our space, emerging startups were threatening our core business. DevOps and the new service architecture enabled them to innovate and release software quickly. We realized that in order to stay competitive we needed to deliver faster.
The next few years were challenging: breaking our monolithic application and transforming our culture into a DevOps oriented one was not easy. Eventually we made it. Our teams became nimble and quick, were able to deliver rapid changes to our code, codified everything in GitHub and as a result, the company was able to release products much faster.
For a while, everything was going well, but slowly things started to go south. We experienced severe inefficiencies and an increase in production issues. We realized that what had happened was that we had lost control over our Git environment. The number of services, functions and git repositories were scary. Our ration was 6 repos per developer. The number of commits and pull request was growing constantly.”
As a cloud computing enthusiast, and a big believer in the unavoidable change the software world is going through, this story baffled me. The DevOps transformation story is supposed to be a positive tale of freedom and agility.
Yet, this executive faced a serious obstacle, one that is mostly unknown in the DevOps chatter. DevOps, Agile and continuous delivery come with a cost, the cost of losing control over your application’s technology stack and development practices.
Git has become the single source of truth
Agile and DevOps broke down software organizations into small autonomous teams that use diverse programming languages, multiple repositories, and numerous microservices. This change has facilitated a faster, more efficient and creative software development. Developers are doing a huge amount of changes on a daily basis. Research done by Jez Humble, Gene Kim and Puppet Labs found that “high-performing” organizations ship code thirty times faster, have half the number of failed deployments, and restore service 12 times faster.
Almost by accident, Git has become the essential tool that software companies use to manage almost everything. Companies are codifying everything they can, with more and more yaml files managing configurations. Modern developer tools use declarative language, too. With ‘infrastructure as code’, even servers are software and are defined in Git repositories. Building, deploying, and managing websites can be done through Git with Netlify.
Increasingly, companies are using Git as their ‘single source of truth’ for their code, configurations and infrastructure. All their business logic lives in Git, and automated processes can turn their Git repositories into built and deployed software or Kubernetes-controlled Docker images. We’ve entered the world of GitOps.
GitOps is all about “pushing code, not containers,” said Alexis Richardson, Weaveworks CEO and chair of the Cloud Native Computing Foundation‘s Technical Oversight Committee, who invented the term. Many companies are doing GitOps without even knowing it, and are struggling with its challenges. These include:
- Visibility: Today’s applications are assembled from hundreds of code components such as Cloud Services, 3rd party APIs, Open Source, Internal packages, etc. Everything is stored in numerous git repositories. Software teams often have no idea what code components other teams are using or who’s working on what.
- Control over changes: Continuous Delivery causes a constant flow of changes into production via an automated software production line with little control. In many cases, companies don’t know if they have branch protection from direct code pushes. Development practices become optional because there’s no way to apply them without excessive control.
- Deployment risks: A small mistake can quickly become a production problem. Forgetting to remove secret keys, adding a .gitignore file or pin dependencies, or even a tiny typo in Ansible Playbook can find its way into production automatically and cause huge problems.
Companies are facing these challenges already, and they’re only getting worse. CIO and developers are buried in repositories and code components and they need to take back control.
datree.io — GitOps policy engine in GitHub
The datree team realized that the best way to deal with this kind of chaos was to bring some order to it at the Git level.
Datree seamlessly connects to an organization’s GitHub and dynamically builds a catalog of all the code components, repositories, committers, and how they all connect to each other. Combined with this is a policy engine of internal development practices to prevent policy violations before they are committed.
Datree’s smart policy engine integrates with any GitHub account and acts as a pull request “gatekeeper”, ensuring developers are aligned with the organization’s and team best practices and standards.
Datree is the first tool to give true visibility, insights and control at the Git level. It balances developer freedom with a company’s need to impose policies and best practice. Organizations are already doing GitOps and are only now beginning to realize it. Datree’s source control management tool is the missing piece that reconnects autonomous teams and brings back control to the modern software enterprise.
Investment in datree.io
I first met Shimon Tolts, the CTO of datree.io, at a small Turkish restaurant. Over hummus, we discussed DevOps and the future of cloud computing. Back then Shimon was the GM of IronSource’s infrastructure and big-data division. Two years later, when Shimon decided to leave his position at IronSource and pursue his new venture, it was natural for the both of us to start working together.
Shimon introduced me to his partners, Arthur Schmunk, who was running business development for AWS in Israel, and Eyar Zilberman, a bright self-made developer from ironSource. The three leading DevOps engineers formed datree.io with a mission to help organizations, that have transitioned or are transitioning to DevOps, to better control code changes and prevent production mishaps.
We were captivated by this group of founders, all of whom are exceedingly intelligent, humble and hard workers. For us, every investment is the beginning of a journey with unexpected turns and obstacles, one that we would never set out to without feeling fully confident in our partners.
We welcome Arthur, Shimon, Eyar and the datree.io team to our portfolio and wish them good luck!