Featherless AI
About the Role
Featherless.ai is building the world’s most reliable and comprehensive open-model inference platform — the infrastructure powering the next generation of AI creators, startups, and enterprises. Our serverless approach to inference unlocks the best GPU utilization in AI infrastructure.
We’re hiring Senior Software Engineers to support and evolve the API gateway to our inference cloud, which is responsible for
-
authentication and inference to all models
-
subscription management and subscription entitlement (e.g. context-length, concurrency limits)
-
and providing the necessary API surface for applications and builders
API Gateway is constantly evolving in response to the unending stream of new models, modalities, clients and inference load.
What you’ll do
The API gateway is managed by the Platform Team, who aim to make Featherless the best place to find and use models. As a member of the platform team, you will
-
undertake feature development and bug fixes to keep up with clients, resolve user issues, and onboard new models
-
improve the reliability of the existing API (increasing instrumentation and monitoring, right-sizing infrastructure)
-
respond to availability incidents
-
triage and resolve issues of inference quality and reliability
-
manage the infrastructure on which our gateway runs
What you’ll bring
-
first-hand experience of the user’s we’re building for (familiarity with popular open LLMs, common clients, and experience building with LLM)
-
experience with the technologies and paradigms of the web (REST, websockets, DNS, networking, opentelemetry)
-
experience with significant components of our stack (k8s, node, mikro-orm, fastify, redis, mongodb, python, elastic cloud, cloudflare, sentry, otel)
-
ability to debug complex issues across a wide stack and build instrumentation as necessary
-
desire to work collaboratively as part of a skilled team
-
Alignment with team and company values, including
-
bias to action
-
responsiveness to users (bug-fixes over features)
-
instinct to iterate
-
subscribing to that done means proven by usage data
-
Other
This team operates on Eastern Time. We are remote, but with a preference to hire in Toronto, Canada.
Originally posted on Himalayas
To apply for this job please visit himalayas.app.
Working in Canada
Canada is a country in North America. Its ten provinces and three territories extend from the Atlantic Ocean to the Pacific Ocean and northward into the Arctic Ocean, making it the second-largest country by total area, with the longest coastline of any country. Its border with the United States is the longest international land border. The country is characterized by a wide range of both meteorologic and geological regions. With a population of over 41 million, it has widely varying population densities, with the majority residing in its urban areas and large areas being sparsely populated. It
More jobs at Featherless AI
Keep exploring on Get A Job.ai
Not quite the right fit? Your next opportunity is a click away.
- Browse all jobs
- More jobs by category
- Remote jobs you can do from anywhere
- Research typical pay for this role
- Set a job alert so new matches reach you first
- Upload your resume to apply faster
Hiring instead? Post a job and reach candidates searching right now.