Methods Visit our website

Intelligent forecast

Socratic's intelligent forecast uses machine learning—specifically, Monte Carlo simulation—to predict completion dates for any body of work.

Monte Carlo is a statistical technique used to model complex systems across a wide range of fields—traffic control, stock predictions, and nuclear reactors, to name a few. The idea is to use the law of large numbers to predict how a complex system might behave. Monte Carlo runs scenarios using our best empirical data and knowledge of the "rules of physics" for the system at hand. By collecting results from enough randomized scenarios, we can get an idea not only of what is most likely to occur, but also visualize the range of possible outcomes.

The basic steps and inputs are as follows:

  1. Set up N tasks with random (but representative) durations.

  2. Set up a pool of M people to complete the tasks.

  3. Assign the N tasks to the M people.

  4. Determine the completion dates for each task.

  5. Construct a burndown chart from these dates.

  6. Repeat steps 1-5 Q times.

In creating a body of work, you provide the number of tasks (N) and the people that will work those tasks (M); Socratic does the rest. This includes randomly-sampled task durations based on your organization’s historical actuals, which is needed for the first step. In other words, the data is entirely personalized to your teams.

We’ve set the number of model runs (Q) to 1,000. The more you do, the more statistical significance you gain, but it also starts to absorb some serious compute cycles. In our own analysis, a thousand runs strikes a nice balance between system responsiveness and forecast reliability. Simply put, this model reveals all of the possible durations if the same body of work was delivered by the same people, a thousand times.

With Monte Carlo simulation, we’re able to absorb a number of complexities that impact delivery and that are otherwise invisible to the naked eye. These include nuances like:

  • The rate at which tasks fall idle;

  • The number and relationship of blocked tasks;

  • The amount of rework involved before a task is completed;

  • The amount of inevitable scope creep in a project;

  • The impact of holidays on people’s availability;

  • The participation of people with specialized skills.

For active plans and objectives, we produce a timeline graph that shows two things:

  1. A completed tasks line, rising to meet a scope line: that is, a burn-up;

  2. A forecast for time to finish, based on your historical actuals. (Monte Carlo actually delivers a likely date range, with outliers on the short and long side of the forecast, and a midpoint date. The midpoint date is what we use for display.)

FAQs

Does Socratic expect any inputs on task size/complexity, e.g. story point values?

No. Our model is concerned only with historical actuals, not manual guesstimates like story points. The inevitable differences in task size or complexity is solved through the law of large numbers (LLN). This also means that no special effort is required to make your tasks approximately the "same size." You simply break down your work as best you can, and leave the rest to us!

How do you account for work that has not yet been assigned?

When a body of work has unassigned tasks, we use an "average contributor" for the model—meaning, we look at your organization's historical average time to complete. As tasks are assigned, our model then personalizes to the assignee(s).

Note that in cases where some tasks are assigned and others are unassigned, our model assumes that the known assignees will also be the people to deliver the unassigned work. This means the model determines when the assigned tasks will be completed by person, and assigns the unassigned tasks based on when each person will have capacity.

As an example: say an objective has 10 associated tasks. Only one of the 10 tasks is currently assigned. The model will build a forecast by assuming the known assignee must first complete their current assigned task, and then pick up the remaining nine tasks and work them sequentially. As tasks are assigned out to additional people, the model adjusts automatically, using the number of known assignees as the available pool of people to deliver the remaining unassigned tasks.

What assumptions do you make about available working time?

Our model removes the following from available working time when building forecasts:

  • Weekends (Saturday and Sunday);

  • U.S. Federal holidays;

  • 18:00—08:00 U.S. PST. (I.e., working hours are 08:00—18:00 U.S. PST.)

Last updated