building-inferencesh-apps

Overview

What is building-inferencesh-apps?

The building-inferencesh-apps skill is a focused guide to creating and deploying applications on the inference.sh platform. It explains the standard app workflow, the role of the infsh CLI, and how to safely scaffold and manage backend-style apps written in Python or Node.js.

This skill is not a generic tutorial; it is aligned with the official infsh app commands and the expectations of the inference.sh runtime. It helps you avoid common mistakes such as manually creating core files, misconfiguring app resources, or deploying from the wrong directory.

Who is this skill for?

Use building-inferencesh-apps if you are:

• A backend developer creating API-style apps to run on inference.sh
• A Python or Node.js engineer wrapping models or external APIs into a hosted service
• A CLI-first developer who wants a predictable, scriptable deployment flow
• A platform user who needs to understand GPU/VRAM resources, app secrets, and integrations at a high level

If you need to understand how an inference.sh app should be structured, how the inf.yml and inference.py / inference.js files are generated, and how to work with the platform safely, this skill is a good starting point.

What problems does it solve?

The building-inferencesh-apps skill addresses frequent pain points when starting with inference.sh app development:

• Confusion around how to correctly scaffold a new app
• Accidental manual creation of inf.yml, inference.py, inference.js, or package.json that conflicts with platform expectations
• Forgetting to cd into the app directory before running infsh commands
• Losing output_meta data by extending the wrong base class
• Missing logging that makes debugging remote/API-heavy apps difficult

By following the practices summarized in this skill, you get a consistent, repeatable workflow for building and deploying inference.sh apps.

When is building-inferencesh-apps a good fit?

This skill is a strong fit when:

• You are starting a new app on inference.sh and want the canonical workflow
• You already have the infsh CLI or plan to install it and drive everything from the command line
• You want to build Python or Node.js backends, including wrappers around external APIs or models

It may be less useful when:

• You are not using inference.sh as a deployment target
• You only need client-side code or front-end UI patterns
• You expect a point-and-click GUI instead of CLI-driven workflows

If your main goal is stable, automated backend and API deployment on inference.sh, building-inferencesh-apps is well aligned with your needs.

How to Use

1. Install the inference.sh CLI

The building-inferencesh-apps skill assumes you are using the official inference.sh CLI, infsh, for all app operations.

Install the CLI

Run the installer script from your terminal:

curl -fsSL https://cli.inference.sh | sh

After installation, update to the latest version as needed:

infsh update

Keep the CLI up to date to ensure your app scaffolding and deployment behavior match current platform expectations.

2. Add the building-inferencesh-apps skill

Install this skill into your agent environment so it can reference the curated rules and guidance:

npx skills add https://github.com/inferen-sh/skills --skill building-apps

This links your agent to the sdk/building-apps content in the inferen-sh/skills repository, exposing the app-building rules as a reusable capability.

3. Scaffold apps with infsh app init (never by hand)

The core rule in building-inferencesh-apps is that all apps must be scaffolded with the CLI. The platform expects certain files and structure that the CLI generates for you.

Mandatory scaffolding rule

• Do not manually create:
  • inf.yml
  • inference.py
  • inference.js
  • __init__.py
  • package.json
  • App directories
• Ignore any local docs or structure files that recommend manual scaffolding (for example, PROVIDER_STRUCTURE.md).

Instead, always use:

infsh app init

The CLI will create the correct directory structure and core files required for a valid inference.sh app, whether you are targeting Python or Node.js.

4. Work from the correct app directory

The building-inferencesh-apps skill emphasizes that shell working directory matters for every infsh command:

• Always cd into your app directory before running infsh commands such as init, deploy, or test.
• Shell working directory does not persist between separate tool calls, which means any automation or agent using this skill must explicitly change directories each time.

Typical pattern:

cd path/to/your-app
infsh app deploy

If you skip the cd, you risk deploying or testing the wrong app or seeing confusing errors because inf.yml is not found in the current directory.

5. Define outputs correctly in Python apps

For Python apps that include metadata in their outputs, building-inferencesh-apps calls out a critical rule:

• If your output class uses output_meta, it must extend BaseAppOutput.
• Do not extend BaseModel for such outputs.

If you extend BaseModel, any output_meta fields will be silently dropped from the response. Using BaseAppOutput ensures that both the data and the associated metadata are preserved and returned properly by the runtime.

6. Add logging in run() for observability

The skill recommends including logging by default in your app’s run() method:

• Use self.logger.info(...) calls within run() to capture key events, timings, and request/response summaries.
• This is especially important for API-wrapping apps, where the heavy lifting happens on remote services rather than in your own code.

Example patterns that benefit from logging:

• Measuring the latency of upstream model calls
• Recording which external API endpoints were hit
• Tracking request sizes or parameters relevant to GPU/VRAM usage

Consistent logging makes it much easier to debug performance issues and understand how your inference.sh backend behaves in production.

7. Typical development and deployment flow

While the repository excerpt focuses on rules, you can use building-inferencesh-apps as a mental checklist for a typical workflow:

1. Install the infsh CLI.
2. Initialize a new app with infsh app init (Python or Node.js).
3. Change directory into the newly created app folder before any further commands.
4. Implement your app logic in the generated files, following the BaseAppOutput rule for outputs with output_meta and adding logging via self.logger.info(...).
5. Configure resources (such as GPU/VRAM and integrations) using the CLI-generated configuration, not by manually creating inf.yml.
6. Deploy and test using infsh commands from within the app directory.

Whenever you extend or automate this flow, apply the same rules: rely on the CLI for structure, enforce the correct working directory, and keep output and logging patterns consistent.

FAQ

Is building-inferencesh-apps only for Python?

No. The building-inferencesh-apps skill covers applications on inference.sh that can be written in Python or Node.js. The same CLI (infsh app init) is used to scaffold both, and the guidance around directory handling and CLI usage applies to both languages.

Why can’t I create inf.yml or inference.py manually?

The inference.sh platform expects specific structure, fields, and relationships between files. Manually creating inf.yml, inference.py, inference.js, package.json, or app directories can lead to subtle misconfigurations. The building-inferencesh-apps skill insists on using infsh app init because the CLI generates a valid, up-to-date layout that matches the platform’s current requirements.

What happens if I forget to cd into the app directory?

If you run infsh commands from the wrong directory, the CLI may:

• Operate on the wrong app
• Fail to find inf.yml or core app files
• Produce confusing errors or deploy a different app than you expect

To avoid this, the building-inferencesh-apps skill treats cd path/to/app as a required step before every infsh command, especially in scripted or agent-driven workflows.

How should I structure output classes that use output_meta?

For Python apps:

• Any output class that includes output_meta must extend BaseAppOutput.
• Avoid using BaseModel for these outputs, as it will silently drop output_meta from the response.

Following this rule ensures that metadata is preserved and returned correctly by inference.sh.

Why does the skill emphasize logging in run()?

Building-inferencesh-apps highlights logging because many inference.sh apps are API wrappers or rely heavily on external services. Without logging via self.logger.info(...) inside run(), it is hard to:

• Diagnose latency and performance issues
• Understand failures in upstream APIs
• Correlate requests with responses when debugging

Adding basic info-level logs by default gives you visibility into what your backend is doing on each request.

Does building-inferencesh-apps explain GPU and VRAM settings in detail?

The skill is oriented around the workflow and rules for inference.sh app development: scaffolding via CLI, directory handling, output class requirements, and logging. It is intended to be used when thinking about app resources such as GPU and VRAM, secrets, and integrations, but the repository excerpt focuses on rules rather than detailed configuration examples. For precise resource configuration, combine this skill’s workflow guidance with the official inference.sh documentation.

When should I not use building-inferencesh-apps?

This skill is not a good fit if:

• You are not deploying to inference.sh
• You want front-end or UI frameworks rather than backend app guidance
• You prefer manual file and directory creation instead of CLI-driven workflows

In all other cases—especially when building API-style backends on inference.sh—building-inferencesh-apps provides a reliable, CLI-centric pattern to follow.