Autoresearch for the real world starts with Open Calls
Open Calls on Asks already let many people try the same brief, checked against the same rules. Recording what worked could make each call improve the next.
Preview: draft, not published. Only a preview or pnpm dev shows it; on Asks this address is not found until the post is published.
Key takeaways
- An Open Call invites many entries for the same brief, and each entry can take a different approach.
- Checking an entry against the rules, choosing a winner and deciding whether a method is worth using again are separate decisions.
- In future, a later call could carry forward the evidence and lessons from earlier attempts, including those that failed.
- Software work could be a fast, affordable first place to test whether this learning loop actually improves results.
On this page6 sections
How could we make solving real problems a cumulative process? Start with an outcome, invite different ways to achieve it, test them in the world, and use what we learn to shape the next attempt. Open Calls on Asks are a practical starting point for this kind of autoresearch.
I think of it as an extension of an ask. An ask defines what counts as done before work begins. An Open Call lets many people try the same brief, each with their own AI and their own approach. Add a memory of what worked, and each attempt could improve the way we act the next time.
What would an Open Call put to the test?
Suppose the call asks for a useful software change in a real repository. The outcome is a change the maintainer accepts, without a material regression, at a cost and review effort worth repeating. One entry might come from a single coding agent working end to end. Another might pair a builder with an independent critic. A third might have an agent write tests first, then a developer handle the difficult integration.
Each entry is a test of one way to produce the outcome. People choose whether to enter and how to do the work. Today, an assistant connected to Asks can read a call and draft an entry, which the person reads before it is sent. The call gives these different approaches a common question to resolve.
- Ask. Publish the brief, the rules entries must meet, the deadline and what the call pays.
- Act. Let different people and methods attempt the work.
- Show evidence. Each entry carries what was delivered and what happened.
- Decide and pay. AI checks each entry against the rules. The poster chooses the pick and any runners-up, who are paid their prizes. A call can also set "Each approved entry": a stated payment for every entry that meets all the requirements, up to an entry limit. A hire call skips the winner prize and keeps the entry payment.
- Learn. The step still to build: use the results to write the next call and improve the approaches worth testing again.
The first four steps work on Asks today. The cycle can begin with a small change and a short deadline. Software gives us fast feedback because a patch can be run against tests and inspected by a maintainer. That makes it a useful first test of the mechanism, even though passing tests alone cannot establish that a change is good for its users.
Decisions need different evidence
A convincing entry deserves attention. It does not establish that its method works in general. We need to keep the decision about what one entry achieved separate from the decision to use its method again.
| Decision | Question | Evidence needed |
|---|---|---|
| Check the entry (today) | Did this attempt meet the call's rules? | The delivered work and the AI check against each rule, with its reasons. |
| Choose the winners (today) | Which entries does the poster want to reward? | The checked entries and the poster's own judgement. |
| Decide what to carry forward (future) | Does the result justify another test or a wider use? | Outcomes across attempts, total cost, failures and the context in which each result occurred. |
Asks already gives a call a published brief, entries, a check of each entry against the rules, and the poster's choice of winners. How verification works explains how a completion is checked against an ask's requirements. The research loop is not built yet. It is a direction to build and test: a call would need a record of which earlier call or method it grew from, what changed, and whether the measured outcome held up in a new setting.
On an Open Call today, the verifier scores how well each entry meets the requirements, out of 100. It is not a measure of real-world impact or a prediction that this method will win again. We should not turn it into either by changing its label.
What makes the next call better?
Imagine the coding method with the best first result costs too much reviewer time. Another method delivers fewer accepted patches, but its failures are easy to diagnose and repair. In future, a team could copy the first call, keep its planning, add the second method's tests and try that combination on fresh work. The new call would name the call it came from and explain what changed. It could also invite a wholly different approach.
This is where the evolutionary idea becomes useful. Variation comes from the entries. Selection comes from decisions tied to evidence. The work itself tests those ideas against a real outcome. Inheritance would happen when a later call names what it took from an earlier one and what it changed.
An unsuccessful approach can still teach us something valuable: a hidden dependency, a poor requirement, a test that missed the real problem, or a cost nobody counted. We can retire a method without treating the person or team behind it as a failure. Good records let the next participant build on that learning.
For a first software trial, I would look for improvement on new tasks, with all attempts and human review time counted. If the selected method only improves on tasks it has seen before, we have learned to fit a benchmark. If it helps another team produce accepted work with less total effort, we have evidence that the learning is travelling.
Can this move beyond software?
Yes, if the outcome can be observed and the people affected can take part on fair terms. A local business might ask for a reliable way to check whether its products are available where customers expect them. Different entries could take different ways to recruit local contributors, guide the work and assess the evidence. The useful result is an accurate, timely check at a cost the business would pay again, with contributors paid for accepted work.
Such a call would need to measure errors, missed locations, contributor effort and repeat demand. A cheap result that excludes difficult places is a poor result. A decision record can show why a completion was accepted, while further observation tells us whether the service actually helped the business. Those are related claims, and they require different evidence.
The same discipline applies to larger aims. We should be able to say what changed, for whom, at what cost, and how sure we are. Where the result is uncertain, the next call can be designed to find out. The ambition is a system in which each attempt makes future cooperation more capable.
FAQ
What is autoresearch for the real world?
It is a process for trying different ways to achieve an outcome, assessing what happened, and using those results to improve the next attempt. Open Calls can invite the approaches and check each one against the rules today; recording the lessons for the next call is still to be built.
Is every Open Call an experiment?
No. A call may simply seek a good entry for a defined job. It becomes part of a research loop when its outcome is measured, its lessons are recorded, and a later call can use them.
Does the highest scoring entry become the winner?
No. The poster chooses the winners. The verifier scores how well each entry meets the requirements, out of 100, and the scores rank the entries that meet all of them. The score alone cannot establish which method will work best in the future.
What happens to entries that do not win?
That depends on how the call pays. A call can offer a prize for the pick and runners-up, and it can also pay each approved entry a stated amount, up to an entry limit. An entry that does not win is still paid if it meets all the requirements and falls within that limit. For autoresearch, a poster could fund qualifying experiments through "Each approved entry" and separately offer a prize for the strongest outcome, so an approach need not win to be paid for accepted work. A failed attempt is not paid automatically. To fund a useful negative result, the brief must make that evidence one of the requirements. Today these are the payment options. Recording what each method taught and using it to write the next call is the learning loop, which is not built yet.
Beyond payment, their methods can be changed or set aside, while useful lessons can inform the next call. Participants should know in advance what will be recorded and reused, especially when their work or information is private.
Start with a real outcome
The first test could be a small software trial: publish a bounded outcome, fund different attempts, measure the full cost of accepted work, and use the evidence to shape a second call. If you build agents that can help people take part, see how agents work with Asks. That is where autoresearch for the real world can begin: ask, act, show evidence, decide, pay and learn.