Skip to main content

Jobe: REST API

Version 1.0 — 17 March 2015 [2]
Author: Richard Lobb [2]

1. Introduction​

This document proposes a RESTful API for sending small student-exercise-type jobs to a "Job Engine" server, called Jobe[4].

A job consists of a source program in a specified language together with possible additional files to be placed in the same directory as the source program[5]. Jobe compiles and/or executes the given job and returns the status of the job plus any output generated[6].

General Intent

This API is similar in general intent to that of the Ideone API (see https://ideone.com/files/ideone-api.pdf)[7]. The major difference is that it is REST-based rather than SOAP-based[8]. Also, it supports the uploading of additional support files, which may be either extra code, such as classes or modules, or may be run-time test data[9].

Jobe has been developed for use in the Moodle CodeRunner question-type plug-in (see https://github.com/trampgeek/CodeRunner)[10]. In this context, the Jobe server is expected to be a custom server behind the Moodle institutional firewall and Jobe will receive requests only from the Moodle server by means of firewalling of the server or other IP whitelisting mechanisms[11].

In more general contexts, authentication and/or authorization can be enforced by higher-level protocols that are not part of this API specification[12]. For example, the current implementation of Jobe uses Ellis Lab's CodeIgniter framework (see http://codeigniter.com) plus the REST-server extension written by Phil Sturgeon (see https://github.com/philsturgeon/codeigniter-restserver)[13]. The latter provides for Basic or Digest HTTP authentication, IP whitelisting, and an API-key mechanism that requires all HTTP requests to include an X-API-KEY header and a known authorisation key[14]. A Jobe administrator can enforce any combination of these mechanisms on top of the API described in this document[15].

2. Requests​

The following table lists all the REST requests[17]. The meanings of the possible response codes are documented in section 3[17].

Content Type

All POST and PUT requests have a content type of application/json; the request body is a JSON-encoded object in which the fields are the parameters specified in the table[17, 18].

Request NameHTTP TypeTarget ResourceParameters (* denotes required)Possible Response CodesComment
submit_runPOST/runsrun_spec*200 OK
202 Accepted
400 Bad Request
404 Not Found		A return code of 200 is accompanied by the run result; 202 denotes the job has been queued for later execution (notes 1, 2, 6)[19].
get_run_statusGET/runresults/idNone200 OK
204 No Content
400 Bad Request
404 Not Found		A return code of 200 is accompanied by the run result; 204 denotes the job is still pending (notes 2, 6)[19].
get_languagesGETlanguagesNone200 OK
400 Bad Request		Returns a JSON-encoded list of supported languages (note 3)[19].
put_filePUT/files/uniqueidfile contents*204 No Content
400 Bad Request
403 Forbidden		It is the client's responsibility to ensure a unique file ID (notes 4, 6, 7, 8)[19].
post_filePOST/filesfile_contents*200 OK
400 Bad Request		Add a file to the collection and get back a unique file identifier (notes 4, 6, 7)[22].
check_fileHEAD/files/uniqueidNone204 No Content
400 Bad Request
404 Not Found		Used by the client to see if the server (still) holds a particular file (notes 6, 8)[22].

Notes​

1. The mandatory run_spec parameter​

The mandatory run_spec parameter is a JSON-encoded job record with the following allowed fields (* denotes a mandatory field/key)[24]:

  • language_id*: The computer language ID of this particular run (see note 3)[25].
  • sourcecode*: The program to compile and/or run[25].
  • sourcefilename: The name to assign to the source code file in the run directory Clyde[26]. If omitted, Jobe either uses a generic language-specific filename like prog.cpp or prog.py, or, where it matters (e.g., Java) a name is inferred from the source code[27]. The latter process is not completely reliable (it uses regular expression matching rather than a full parse) so for such languages an explicit sourcefilename is strongly recommended[28].
  • input: The standard input data, if required[29].
  • file_list: A list of (file_id, file_name) pairs, specifying which files should be loaded into the run-time directory[30]. The file_id is the unique file identifier as supplied in put_file and file_name is the name assigned to that file when it is loaded[31].
    Extension Note

    As a possible extension, a triple of (file_id, file_name, is_source) might be permitted instead of a pair; the third parameter is true if the specified file is a program source file that should be included in the compile-and-build operation in a language-dependent manner[32, 33].

    File identifiers are required to be purely alphanumeric and at least 8 characters in length[33]. Filenames must be made only from alphanumeric characters plus -, _ and .[34].
  • parameters: A server-dependent set of (key, value) pairs (i.e., a JSON record), which might include things like maximum execution time, maximum memory usage, compile flags[35].
  • debug: If provided and true, the server is invited to include extra debugging information in the response or to retain extra information itself for later inspection[35]. Server-dependent[36].

2. Run Execution Modes​

The API allows for either an immediate run of a submitted job, returning the run results in the response, or a deferred run in which the server simply accepts the job and enqueues it for later execution[37]. The server has discretion over which of these modes to use and the client must respect that choice, polling for a result if the run submission returns 202 Accepted rather than 200 OK[38].

The immediate mode is preferred for fast turn-around jobs on relatively lightly-loaded servers while the deferred mode would be required under high load[39]. The server may choose to use a mix of the two depending on load and/or expected maximum execution time[40].

3. Language List​

The returned language list is a JSON-encoded list of (language_id, language_version) pairs of all supported languages[41]. The client must use one of the returned language_ids in all its run submissions[42].

The formats of the language_id and language_version are server-dependent: they are just strings[43]. However, they should preferably be human-readable, e.g., ('C99', 'gcc 4.8.1'), to allow clients to present the language list to on-line users[43, 44]. The language_id must be unique; if multiple versions of a language are supported, each must have its own ID[44, 45].

4. Shared-Server Context & PUT Requests​

The use of PUT to place a specific file on the server is not satisfactory in a shared-server context where the clients might not be trusted to provide a globally unique file ID[46, 47].

Security Warning

If a server does not trust the sender it should return 403 Forbidden and the client must then fall back to using post_file instead[48].

5. Idempotency & Result Holding​

Ideally, if the get_status request is to be idempotent, the server should keep run results for some "reasonable" time[49]. However, the holding time from when the results are first returned to the client to when they are discarded from the server is at the discretion of the server, and may be zero[50].

6. Server Caching of Support Files​

The file interface supports server caching of support files, which might be large test data files that one does not wish to upload multiple times[51].

The recommended approach is that used by the CodeRunner client: attempt a submit_run without uploading any files and if the response is 404 Not Found, upload all the files using put_file and try the run again[52]. Alternatively, if it is unlikely that the files have been uploaded already, they can be uploaded before the run[53].

7. File Contents Parameter​

The file_contents parameter to put_file and post_file requests is a string containing the file contents (which might be binary) encoded in standard base_64[54]. The server decodes the file_contents before writing it to the file cache[55].

Validation Error

If the contents are not a valid base-64 encoding, 400 Bad Request is returned[56].

8. File Cache Persistence​

The file cache is, as the name implies, simply a cache rather than a permanent file store[57]. Persistence of a file beyond a few hours is not guaranteed[57, 58].

It is thus unsafe to use a check_file request to confirm the existence of a required file prior to the run that requires it[59]. The file might be deleted from the cache between the two calls[59, 60]. See note 6 for the recommended approach[60].

3. Meanings of the Different Response Codes​

This section describes the meanings of the various possible HTTP response codes returned by the requests listed in section 2[61, 62].

Unlisted Method Code

405 Method Not Allowed is the only response code not explicitly listed in section 2, because it is the one issued when a request does not match any of the specified requests[62].

  1. 200 OK: Returned by a successful request that is accompanied by some response data[63]. Not all successful requests return 200 OK, as follows:
    • submit_run returns 200 together with the full run result as in section 4 below if the run can be done immediately, but returns 202 Accepted together with a run_id if the run is queued for later execution[64].
    • get_run_status returns 200 OK plus the run result if the job is complete but returns 204 No Content if the job is still queued[65].
  2. 202 Accepted: Returned by the submit_run request if the server has enqueued the job for subsequent execution rather than running it immediately[67]. The response data is then a unique run_id for use in subsequent get_run_status requests[68].
  3. 204 No Content: Returned by the get_run_status request if the request relates to a pending job for which the result is not yet available[69]. It's also returned by check_file if the file exists and by a successful put_file[70].
  4. 400 Bad Request: Returned by any request that has syntactic or semantic errors in any of the parameters or is missing a required parameter[71].
  5. 403 Forbidden: Returned by put_file if the server does not trust the client to provide a unique file ID, in which case the client must use the post_file command instead, where the server provides file IDs[72].
  6. 404 Not Found: Returned by any request to a resource collection other than those listed, or by a request for a specific unknown resource, or by a submit_run request containing unknown file IDs[73].
  7. 405 Method Not Allowed: Returned by any request to a resource or resource collection that uses a method that is not defined for that RunResult resource (collection)[74].
  8. Other Return Codes: Other return codes, such as 401 Unauthorized, may also be returned if higher-level authentication and authorisation protocols are enabled, as briefly discussed in the introduction[75].

4. The RunResult Object​

The data returned with a 200 OK response to either a submit_run or get_run_status request is a JSON-encoded record with four fields, as follows[77, 79]:

  1. run_id: The unique ID of this particular run (which may or may not be usable in a subsequent get_job_status request; see section 2, note 5)[80].
  2. outcome: The outcome code of the job, mapping to the values below[81].
ValueMeaning
11Compilation error. The cmpinfo field should offer further explanation[82].
12Runtime error. The job compiled but threw an exception at run time that isn't covered by any of the more-specific errors below[82].
13Time limit exceeded. The job was killed before it ran to completion as a result of the server-specified time limit (or a possible time limit specified via the parameters field of the job request) being reached[82].
15OK. The run ran to completion without any exceptions[82].
17Memory limit exceeded. The job was killed before it ran to completion as a result of the server-specified maximum memory limit (or a possible memory limit specified via the parameters field of the job request) being reached[82].
19Illegal system call. The task attempted a system call not allowed by this particular server[82].
20Internal error. Something went wrong in the server. Please report this to an administrator[82].
21Server overload. No free Jobe user accounts. Probably something has gone wrong[82].

The precise situations under which these outcome values are returned will be server-dependent[83]. For example, a server might limit the memory by denying memory allocation requests without terminating the job: the job would then possibly generate its own error message and exit without throwing an exception[84]. Similarly, a server might not recognise an illegal system call as such but just lock the job in a chroot jail so that potentially dangerous system calls are unable to do any damage[85].

  1. cmpinfo: Any output generated by the compiler (if there is one) at compile time[86].
  2. stdout: The standard output from the program run[87].
  3. stderr: The standard error output from the program run[88].
Last updated on by Adriano Ruseler