Software Engineer - Distributed Systems

Since 2018, Frame AI has leveraged the technology of Large Language Models (LLMs) to build the essential integration point for understanding customers. Frame AI empowers organizations to truly understand the people they serve. New technology offers an opportunity for processes that are both efficient and humane, by listening at scale. We use AI to pull individual voices from the crowds, recognize meaningful communities within giant aggregations, and power experiences that react to the needs of both customers and the people serving them. By listening better, we make better experiences, better jobs, and better businesses. Frame AI’s institutional investors include Velvet Sea Ventures, G20 Ventures, Greycroft, and FirstMark.

This Role

You will own the effort to transform Frame’s architecture for Data Products into a linearly scalable, cost-effective system. You will increase Frame’s value to customers, especially its growing Enterprise base, by handling and integrating more data. Your work will reduce the time to market to deliver innovative Data Products features that are built for speed and usability from the ground up. The ideal candidate has meaningful experience owning some or all of a significant data architecture involving configurable aggregations and numerical calculations. Candidates with experience in machine learning and natural language domains are preferred. The base pay for this role is $140,000–$225,000 per year based on leveling within either the Senior or Staff Engineer bands. You are also eligible for employee benefits and company equity grants.

What You’ll Do

• Provide technical leadership that informs the Data Products roadmap and guides strategic technology investments.
• Develop a scalable architecture for Data Products based on proven technologies used to replicate data, orchestrate workflows, and perform large-scale numerical calculations.
• Train staff to become effective contributors on the data architecture.
• Invest in all stages of the Software Development Lifecycle around Data Products to ensure alignment to Product goals, technical feasibility, and maintainability.
• Collaborate with other technical leaders to ensure effective integration with data systems and provide technical feedback on company-wide data architecture concerns.
• Design and maintain Data Products APIs consumed by other product teams.

Who We’re Looking For

• Reader of manuals. You have demonstrated success learning unfamiliar technologies and you frequently consult documentation to get the most out of your systems.
• Seeks a startup environment. You appreciate having a high impact role and bringing exciting new technologies to market. You respond constructively to change and team up with others who rise to the challenge.
• Skillful judge of technical risks. You have built up tools across the development lifecycle to protect critical data infrastructure from costly errors.
• A continuous improver. You resist impulsive technical choices while managing a backlog of high-impact projects that you execute steadily and reliably.
• Inclusive collaborator. You view communication as one of your strongest skills, and you seek opportunities within your team and beyond to share knowledge that helps people work better.
• Boundaryless coder. You move across project and dependency interfaces to understand how things work, improve system designs, debug challenging issues, and even to fix a flaw or two in open source libraries.
• At home in the cloud. You jump easily from your local command-line to terminal sessions, and you are comfortable working with AWS.
• Reliable experience. You have built and maintained a data architecture of significant scale and complexity and have a good sense about what works well in different situations.

Requirements

• Excellent communicator across modes from technical designs, daily progress updates, code reviews, and responding to production incidents.
• Expert on data processing strategies including batch, incremental, and streaming systems and their tradeoffs.
• Expert knowledge of columnar data stores, distributed data processing, and ELT orchestration tools, with a preference for experience with AWS RedShift, Apache Spark, and Apache Airflow.
• Proficient with deployment and system architecture concerns required to operate data systems on varying machine types and meeting specific hardware requirements.
• Familiarity with meeting compliance requirements in data systems, especially in regards to data access and removal.
• Strong fundamental engineering skills including proper use of standard data structures and algorithms, reproducing issues and debugging programs, and awareness of systems concerns such as networking and process management.
• Proven experience owning a significant data architecture for 3 or more years consecutively.
• Excellent ability to map product and design requirements to engineering requirements.
• Proficiency with SQL queries and numerical computation using an industry standard programming language.
• Expert operator of data systems including workflow management, system monitoring, and capacity planning.
• Proficient using Python 3 and scientific libraries such as numpy.

Working at Frame AI

Frame AI is an environment where skilled engineers thrive. We embrace development and testing practices that will let you leverage your skills, making far-reaching changes that you can deploy easily and repeatedly. Here, you can exercise and push the limits of your data engineering skills while working with a top-notch data team on challenging, relevant, and high-impact problems. Frame AI is based the NYC region, but this role is open to remote applicants in the US. This role is not eligible for immigration/visa sponsorship in the US.