GitHub Copilot’s cloud agent, called the coding agent or async agent, runs in a full clone of your repository and works through multi-step coding tasks without a developer watching. You assign a task, it opens a pull request with the work done, and you review it. The pitch is clear. The reality, after several weeks of daily use on this project, is more specific than the marketing suggests.
What the agent actually does
The agent gets a GitHub issue or a direct prompt, spins up an Actions runner, clones the repository, and works through the task using a full suite of tools: grep, file reads, bash, the GitHub MCP server for PR and issue context, and a browser for research. It commits changes, pushes a branch, and opens a draft PR. If you add a comment on the PR, it spins up a new run to address the comment.
That is the full loop. There is no interactive back-and-forth, no inline suggestions, and no chat window. The agent either finishes the task in one pass or it finishes it after one or two review-and-comment cycles. The workflow feels closer to delegating to a junior engineer than to pairing with an autocomplete tool.
Where it performs well
The agent is reliable on tasks with a narrow scope and clear acceptance criteria. Adding a new content type to an existing pipeline, fixing a validation function that rejects valid inputs, adjusting a frontmatter parser to handle an edge case – these land correctly on the first pass most of the time. The key is that the agent can read all the relevant code before writing anything, so it does not make the same mistakes a human would make on an unfamiliar codebase in an hour.
It is also genuinely useful for tasks that are tedious rather than hard: reformatting a configuration file, adding a missing error message, updating a test fixture to match a renamed field. These tasks are worth assigning because the agent does them faster than a developer and the review is quick because the scope is contained.
Content authoring is a gray area. The agent can write structured Markdown posts that follow an established format, and it will observe frontmatter conventions, word count requirements, and style rules if they are spelled out clearly in the repository instructions. Whether the output meets your quality bar depends on how specific those instructions are. Vague instructions produce generic content. Specific instructions produce work that is useful, if not always excellent.
Where it falls short
Multi-file refactoring with unclear boundaries is where failures happen. If a task requires understanding an implicit contract between two modules and changing both in a coordinated way, the agent sometimes changes one correctly and misses the other. The failure mode is not a crash; it is a passing test suite with a behavioral regression that only surfaces in manual testing. This is the same failure mode a junior engineer has on an unfamiliar codebase, and the mitigation is the same: write better tests first, then assign the refactoring.
Anything requiring external context not in the repository is also unreliable. If a task depends on knowledge of a third-party API behavior that is not documented in the code, the agent will guess. If it guesses wrong and the code compiles and tests pass, the wrong guess lands in the PR looking like a correct implementation.
The agent also does not self-correct well on the first pass. If it commits a mistake in commit 3 of 8, it typically builds on that mistake rather than reconsidering it. Comments on the PR are the right mechanism for correction, not hoped-for self-review.
The CI cost
Each agent run uses an Actions runner for the duration of the task. Simple tasks take 5 to 10 minutes. Complex tasks with exploration, multiple edits, and test runs take 20 to 40 minutes. At GitHub’s standard per-minute rate, a complex task costs roughly $0.04 to $0.08. Running the agent five times a day on a mix of simple and complex tasks costs under $1.50 per day, which is less than the equivalent developer time and substantially less than an LLM API-heavy workflow that calls the model dozens of times per task.
The hidden cost is review time. A PR that changes 12 files across 3 modules requires more review than a PR that changes 2 functions in 1 file. Keeping tasks narrow reduces both the failure rate and the review burden. The best-performing task assignments are specific enough that the resulting PR can be reviewed in under five minutes.
Practical takeaways
Use the cloud agent for tasks where the acceptance criteria are clear, the scope is contained to one or two files, and a passing test suite is a meaningful signal of correctness. Do not use it for cross-cutting refactors or for tasks where the right behavior depends on context that is not written down anywhere in the repository.
The agent works best as an accelerator for routine engineering tasks, not as an autonomous engineer. The humans on the team still decide what to build and verify that the build is correct. What the agent removes is the tedium of typing out the implementation once the decision is made.
After several weeks of daily use, the agent has shipped more code than it has broken. That is the bar it needs to clear, and it clears it, with the caveats above.
Leave a Reply