prodigy.web.jobs¶

Analysis jobs for the web GUI.

Every analysis runs in its own process (spawn), so that it can be stopped at any time: on request (cancel), when it exceeds the time limit, or when the server shuts down. The child process streams its console output and progress events back through a pipe and finally sends the JSON result (prodigy.analysis.result).

exception BusyError[source]¶

Bases: Exception

Too many analyses are running at the same time.

class Job(id, process, conn, time_limit, summary, created=time(), started=monotonic(), finished=None, state='starting', log='', log_dropped=0, progress=dict(), result=None)[source]¶

Bases: object

Parameters:

id (str)
process (Any)
conn (Any)
time_limit (float)
summary (Dict[str, Any])
created (float)
started (float)
finished (float | None)
state (str)
log (str)
log_dropped (int)
progress (Dict[str, Any])
result (Dict[str, Any] | None)

id: str¶

process: Any¶

conn: Any¶

time_limit: float¶

summary: Dict[str, Any]¶

created: float¶

started: float¶

finished: float | None = None¶

state: str = 'starting'¶

log: str = ''¶

log_dropped: int = 0¶

progress: Dict[str, Any]¶

result: Dict[str, Any] | None = None¶

property elapsed: float¶

property running: bool¶

to_json(log_offset=0)[source]¶

Parameters:: log_offset (int)
Return type:: Dict[str, Any]

class JobManager(*, max_jobs=2, time_limit=120.0, memory_limit=None, keep_finished=32)[source]¶

Bases: object

Starts, watches and stops analysis processes.

Parameters:

max_jobs (int)
time_limit (float)
memory_limit (int | None)
keep_finished (int)

submit(request, time_limit=None)[source]¶

Parameters:

request (Dict[str, Any])
time_limit (float | None)

Return type:

Job

get(job_id)[source]¶

Parameters:: job_id (str)
Return type:: Job | None

cancel(job_id)[source]¶

Parameters:: job_id (str)
Return type:: Job | None

shutdown()[source]¶

Return type:: None

running_count()[source]¶

Return type:: int