Engineering principles at Upflow

At Upflow, we believe that a small, tight-knit team can achieve extraordinary things. By embracing core principles related to simplicity and continuous improvement, we foster a serene and stimulating environment where everyone contributes meaningfully.

Early in our journey, the founding engineering team at Upflow agreed on core principles we would stand by. In this article, we'll explore these principles and how they guide our development process to keep our software robust and maintainable.

You aren't gonna need it

When building something new, it's easy to get carried away and anticipate every possible future scenario. We might ask ourselves:

Will this scale to handle a massive increase in data?

Is this flexible enough to accommodate unforeseen use cases?

While these are valid concerns, prematurely addressing hypothetical issues often leads to:

• Unnecessary complexity: Over-engineering solutions for problems that may never arise can make the code harder to understand, debug, and maintain.

• Wasted effort: Time spent on speculative needs is time not spent on delivering real value to our users.

That’s why we embrace the YAGNI principle. We focus on writing the simplest, most effective code to meet the current requirements. We anticipate potential future needs to avoid costly refactoring projects, but only if we can maintain the same level of simplicity.

• Prioritize present needs: We address the immediate needs of the project, ensuring the code is clean, well-tested, and delivers the desired functionality.

• RFC for critical future-proofing: For scenarios where future changes could lead to significant work (e.g., breaking API changes, major database schema alterations), we propose and discuss potential solutions in a formal Request for Comments (RFC) document.

Prioritize boring technology

Developers are naturally drawn to the latest and greatest technologies – new frameworks, languages, and tools. While exploring cutting-edge solutions can be exciting and intellectually stimulating, it's crucial to prioritize proven technologies that align with our specific needs.

Introducing trendy technologies, even those with great potential, can sometimes introduce unnecessary complexity or risks:

• Unnecessary complexity: Integrating a new technology might add complexity that doesn't directly benefit the project's goals.

• Potential instability: Newer technologies may have unaddressed bugs, lack maturity, or have rapidly evolving APIs, which can lead to unexpected issues and increased maintenance burden.

• Limited community support: Smaller communities may offer limited resources, making it harder to find solutions to problems or get timely support.

On the other hand, established technologies have several advantages:

• Maturity and stability: Proven technologies are generally more mature and stable, with fewer critical bugs and well-defined APIs.

• Strong community support: Large and active communities provide access to extensive documentation, tutorials, and readily available solutions to common problems.

• Rich ecosystem: Established technologies often have robust ecosystems with a wide range of compatible tools, libraries, and integrations, simplifying development and maintenance.

So while we encourage continuous learning and exploration, we prioritize proven technologies for our most strategic needs.

Don’t blindly follow best practices

We value thoughtful decision-making and avoid blindly following "best practices" without careful consideration. While best practices offer valuable guidance, they are not one-size-fits-all solutions. The success of any approach depends heavily on the unique context of the project, including its scale, complexity, team size, and specific requirements.

For example, micro-services architectures were often touted as a best practice and can solve organizational scaling challenges. However, they are not necessarily optimal for a small team like ours as they also introduce new complexities - operational overhead, monitoring, interface versioning, end-to-end testing…

Before adopting any "best practice", we carefully evaluate its suitability for Upflow:

• Understand the underlying principles: Grasp the reasoning behind the best practice and how it addresses common challenges.

• Analyze our specific needs: Evaluate the project's unique requirements and constraints.

• Consider potential trade-offs: Weigh the potential benefits against the potential drawbacks (e.g., increased complexity, impact on team productivity).

By carefully evaluating each "best practice" in this way, we can make informed decisions that fit our specific context and contribute to long-term success.

Don't be too smart

Kernighan's Law states that

Debugging is twice as hard as writing the code in the first place. Therefore, if you write the code as cleverly as possible, you are, by definition, not smart enough to debug it.

This serves as a good reminder that we don't only write code for the machine, but also for other developers who will have to maintain it.

Writing complex code can be justified if it adds a lot of value (e.g., a clever abstraction can get rid of a lot of boilerplate code), but if it doesn't, the only result will be lower team productivity due to the additional time spent understanding or debugging the code.

That’s why we prioritize simplicity over cleverness to ensure that our codebase remains understandable and maintainable over time.

Strive for simplicity, avoid oversimplification

As we’ve seen with the previous principles, simplicity is a core value at Upflow. We believe in finding the most elegant and straightforward solutions to engineering challenges. However, it's crucial to distinguish between essential complexity and incidental complexity.

• Essential complexity: This is the complexity inherent to the problems we solve. Attempting to oversimplify these problems can lead to brittle solutions that fail to address the underlying issues.

• Incidental complexity: This type of complexity arises from poor design choices, unnecessary abstractions, or a lack of focus. It hinders maintainability, increases the risk of errors, and makes the code harder to understand and work with.

At Upflow, we strive to minimize incidental complexity by choosing clear and concise designs, but we also carefully consider the essential complexity of the problems we encounter and avoid oversimplifying solutions that would ultimately compromise their effectiveness.

Work in small batches

We believe in the power of working in small batches. This approach involves breaking down large features into smaller, independently deliverable tasks that can typically be completed within 1-2 days.

By working in this manner, we reap several significant benefits:

• Rapid feedback loops: Small, incremental changes allow us to quickly gather feedback from users, stakeholders, and other team members. This enables us to identify and address potential issues early on, minimizing the risk of costly rework later in the development cycle.

• Reduced risk and improved resilience: When issues arise, the impact of small changes is significantly lower. This makes it easier to identify the root cause of problems and implement quick fixes or rollbacks with minimal disruption to the system.

• Enhanced code quality and maintainability: Small, focused pull requests are easier to review, leading to higher code quality and fewer bugs.

• Flexibility: working in small batches allows us to stop a project earlier if needed without losing the value of the work already completed**.** If a feature proves to be less valuable than initially anticipated, or if priorities shift, we can easily stop development without significant sunk costs.

• Increased efficiency and productivity: Working in small batches forces us to streamline our development and deployment processes. Automation plays a crucial role, enabling us to rapidly and reliably deploy small changes. This frequent delivery cycle provides a sense of momentum, keeping the team motivated and engaged.

Continuous refactoring

Even with careful design and a focus on simplicity, our codebase will inevitably evolve. As requirements change, new features are added, and the scale of our systems grows, certain architectural decisions may become outdated or inadequate.

Therefore, we embrace continuous refactoring as an integral part of our development process.

• Don't fear change: When we encounter an abstraction that has become overly complex, limited, or irrelevant, we don't shy away from refactoring it.

• Leverage automated tests: Our robust suite of automated tests provides a safety net for refactoring efforts. They help us quickly identify and address any unintended consequences of code changes.

• Identify and fill testing gaps: If refactoring exposes missing tests, it's an opportunity to improve our test coverage and enhance the overall quality of our codebase.

By embracing continuous refactoring, we ensure that our codebase remains maintainable and aligned with our evolving needs.

Learn from past mistakes and prevent recurrence

As discussed in the YAGNI principle, we don’t try to anticipate and prevent every possible problem when developing a new feature because it would lead to over-engineering and hinder our ability to deliver value quickly. However, when the same problem occurs multiple times, we cannot simply ignore or tolerate them. It would degrade the quality of our product, negatively impact the user experience, and erode team morale.

Our Approach:

• Prompt detection and resolution: We strive to proactively detect and resolve issues as quickly as possible, considering the impact of the issue and the cost of resolution.

• Root Cause Analysis: For critical issues, we conduct thorough root cause analyses to understand the underlying factors that contributed to the problem.

• Implement preventative measures: Based on our root cause analysis, we implement appropriate preventative measures to prevent the issue from recurring.

Eliminate toil

Software engineers should focus on problem-solving and developing new features, not on repetitive and manual tasks.

• Identify and eliminate toil: Toil refers to any manual, repetitive, and low-value work that can be automated. Examples include:

  • Running the same commands repeatedly.

  • Manually performing data migrations.

  • Tedious debugging and troubleshooting.

  • Repetitive administrative tasks.

• Invest in automation:

  • We strive to automate routine tasks.

  • We utilize tools like CircleCI or Terraform to automate deployments or manage our infrastructure.

  • We continuously improve our logging and alerting to fasten troubleshooting and avoid wasting time on false alerts.

By eliminating toil, we free up engineers to focus on higher-value activities such as building a great product.

Product Mindset

We expect developers to go beyond simply implementing features. We encourage a strong product mindset where engineers:

• Understand the "why": understand the business context and the user needs that drive each feature or project.

• Proactively identify and address gaps: Be proactive in identifying and addressing potential issues or missing requirements, minimizing the need for excessive back-and-forth with the product team.

• Contribute to product decisions: Actively participate in product discussions, offering insights and alternative solutions to find the optimal balance between the business value of a feature and its engineering cost.

Ownership

At Upflow, engineers are accountable for the quality and impact of their work from conception to production. This includes:

• Ensuring code quality: Writing clean, well-tested, and maintainable code.

• Minimizing risk: Considering the potential impact of their changes on the overall system, and proactively identifying and fixing potential issues.

• Delivering value: Focusing on delivering solutions that meet the needs of our users, without losing sight of the importance of delivering within a reasonable timeframe.

Embrace calculated risks

We understand that innovation requires taking calculated risks. We believe that fear of breaking something or disappointing users can significantly hinder our ability to move quickly.

• Prioritize productivity: While minimizing risk is important, we prioritize the productivity of our team to deliver maximum value to our users.

• Learn from failures: We view potential setbacks as learning opportunities.

• Foster a culture of experimentation: We encourage a culture where it's safe to experiment and try new things.

Conclusion

These engineering principles have helped us build our product consistently over the years and we are proud to live by them. If you are curious about them or would like to join the team, don't hesitate to reach out!