Package org.gbif.api.service.pipelines
Interface PipelinesHistoryService
public interface PipelinesHistoryService
-
Field Summary
Fields -
Method Summary
Modifier and TypeMethodDescriptionlong
addPipelineExecution
(long processKey, @NotNull PipelineExecution pipelineExecution) Adds/persists the information of a pipeline execution.void
allowAbsentIndentifiers
(@NotNull UUID datasetKey) Mark latest failed identifier stage as finished and continue interpretation process for datasets were identifier stage failed because of a threshold limitvoid
allowAbsentIndentifiers
(@NotNull UUID datasetKey, int attempt) Mark failed identifier stage as finished and continue interpretation process for datasets were identifier stage failed because of a threshold limitlong
createPipelineProcess
(@NotNull PipelineProcessParameters params) Creates/persists a pipelines process of dataset for an attempt identifier.getPipelineProcess
(@NotNull UUID datasetKey, int attempt) Gets the PipelineProcess identified by the dataset and attempt identifiers.getPipelineStep
(long stepKey) Gets the PipelineStep of the specified keys.getPipelineStepsByExecutionKey
(long executionKey) Gets the PipelineSteps list of the execution key.getRunningExecutionKey
(@NotNull UUID datasetKey) Gets execution key for running datasetgetRunningPipelineProcess
(Pageable pageable) Returns information about all running pipelines executionsLists the history of allPipelineProcess
of a dataset, sorted descending from the most recent one.Lists the history of allPipelineProcess
, sorted descending from the most recent one.void
Mark all pipeline executions as finished to clean running UIvoid
markPipelineExecutionIfFinished
(long executionKey) Mark pipeline execution as finished when all pipelin steps are finishedvoid
markPipelineStatusAsAborted
(long executionKey) Change status to ABORTED and set finished date if state is RUNNING, QUEUED or SUBMITTED, and set pipeline execution as finishedvoid
notifyAbsentIdentifiers
(UUID datasetKey, int attempt, long executionKey, String message) Sends a notification to the data administrators about absent identifiers issues with the dataset.runAll
(@NotBlank(message="Steps parameter is required") String steps, @NotBlank(message="Reason parameter is required") String reason, boolean useLastSuccessful, boolean markPreviousAttemptAsFailed, RunAllParams runAllParams, Set<String> interpretTypes) Runs the last attempt for all datasets.runPipelineAttempt
(@NotNull UUID datasetKey, int attempt, @NotBlank(message="Steps parameter is required") String steps, @NotBlank(message="Reason parameter is required") String reason, boolean markPreviousAttemptAsFailed, Set<String> interpretTypes) Re-run a pipeline step.runPipelineAttempt
(@NotNull UUID datasetKey, @NotBlank(message="Steps parameter is required") String steps, @NotBlank(message="Reason parameter is required") String reason, boolean useLastSuccessful, boolean markPreviousAttemptAsFailed, Set<String> interpretTypes) Restart last failed pipelines step for a dataset.void
sendAbsentIndentifiersEmail
(@NotNull UUID datasetKey, int attempt, @NotNull String message) Deprecated.long
updatePipelineStep
(@NotNull PipelineStep pipelineStep) Update the information of a pipeline step.
-
Field Details
-
STEPS_REQUIRED_MESSAGE
- See Also:
-
REASON_REQUIRED_MESSAGE
- See Also:
-
-
Method Details
-
history
Lists the history of allPipelineProcess
, sorted descending from the most recent one.- Parameters:
pageable
- paging request- Returns:
- a paged response that contains a list of
PipelineProcess
.
-
history
Lists the history of allPipelineProcess
of a dataset, sorted descending from the most recent one.- Parameters:
datasetKey
- dataset identifierpageable
- paging request- Returns:
- a paged response that contains a list of
PipelineProcess
.
-
getPipelineProcess
Gets the PipelineProcess identified by the dataset and attempt identifiers.- Parameters:
datasetKey
- dataset identifierattempt
- crawl attempt identifier- Returns:
- an instance of pipelines process if exists.
-
getRunningPipelineProcess
Returns information about all running pipelines executions -
createPipelineProcess
Creates/persists a pipelines process of dataset for an attempt identifier. If the process already exists it returns the existing one.- Parameters:
params
- pipeline process parameters, contain dataset key and attempt- Returns:
- the key of the
PipelineProcess
created.
-
addPipelineExecution
Adds/persists the information of a pipeline execution.- Parameters:
processKey
- sequential identifier of a pipeline processpipelineExecution
- pipeline execution data- Returns:
- the key of the PipelineExecution created.
-
getRunningExecutionKey
Gets execution key for running dataset- Parameters:
datasetKey
- dataset identifier- Returns:
- running execution key
-
updatePipelineStep
Update the information of a pipeline step.- Parameters:
pipelineStep
- step to be added- Returns:
- the key of the PipelineStep created.
-
getPipelineStep
Gets the PipelineStep of the specified keys.- Parameters:
stepKey
- key of the pipeline step- Returns:
PipelineStep
.
-
getPipelineStepsByExecutionKey
Gets the PipelineSteps list of the execution key.- Parameters:
executionKey
- key of the pipeline execution- Returns:
List<PipelineStep>
.
-
markAllPipelineExecutionAsFinished
Mark all pipeline executions as finished to clean running UI -
markPipelineExecutionIfFinished
Mark pipeline execution as finished when all pipelin steps are finished- Parameters:
executionKey
- key of the pipeline execution
-
markPipelineStatusAsAborted
Change status to ABORTED and set finished date if state is RUNNING, QUEUED or SUBMITTED, and set pipeline execution as finished- Parameters:
executionKey
- key of the pipeline execution
-
runAll
RunPipelineResponse runAll(@NotBlank(message="Steps parameter is required") @NotBlank(message="Steps parameter is required") String steps, @NotBlank(message="Reason parameter is required") @NotBlank(message="Reason parameter is required") String reason, boolean useLastSuccessful, boolean markPreviousAttemptAsFailed, @Nullable RunAllParams runAllParams, @Nullable Set<String> interpretTypes) Runs the last attempt for all datasets.- Parameters:
steps
- steps to runreason
- reason to runuseLastSuccessful
- true if we want to run the latest successful attemptmarkPreviousAttemptAsFailed
- previous status can't be wrong, when CLI restarted during processing a datasetrunAllParams
- parameters, contain datasets to excludeinterpretTypes
- is used for partial interpretation such as only TAXONOMY, METADATA and etc- Returns:
RunPipelineResponse
.
-
runPipelineAttempt
RunPipelineResponse runPipelineAttempt(@NotNull @NotNull UUID datasetKey, @NotBlank(message="Steps parameter is required") @NotBlank(message="Steps parameter is required") String steps, @NotBlank(message="Reason parameter is required") @NotBlank(message="Reason parameter is required") String reason, boolean useLastSuccessful, boolean markPreviousAttemptAsFailed, @Nullable Set<String> interpretTypes) Restart last failed pipelines step for a dataset.- Parameters:
datasetKey
- dataset keysteps
- steps to runreason
- reason to runuseLastSuccessful
- true if we want to run the latest successful attemptmarkPreviousAttemptAsFailed
- previous status can't be wrong, when CLI restarted during processing a datasetinterpretTypes
- is used for partial interpretation such as only TAXONOMY, METADATA and etc- Returns:
RunPipelineResponse
.
-
runPipelineAttempt
RunPipelineResponse runPipelineAttempt(@NotNull @NotNull UUID datasetKey, int attempt, @NotBlank(message="Steps parameter is required") @NotBlank(message="Steps parameter is required") String steps, @NotBlank(message="Reason parameter is required") @NotBlank(message="Reason parameter is required") String reason, boolean markPreviousAttemptAsFailed, @Nullable Set<String> interpretTypes) Re-run a pipeline step.- Parameters:
datasetKey
- dataset keyattempt
- attempt to runsteps
- steps to runreason
- reason to runmarkPreviousAttemptAsFailed
- previous status can't be wrong, when CLI restarted during processing a datasetinterpretTypes
- is used for partial interpretation such as only TAXONOMY, METADATA and etc- Returns:
RunPipelineResponse
.
-
sendAbsentIndentifiersEmail
@Deprecated void sendAbsentIndentifiersEmail(@NotNull @NotNull UUID datasetKey, int attempt, @NotNull @NotNull String message) Deprecated.Sends email to data administrator about absent identifiers issue with a datasetDeprecated: use
#notifyAbsentIdentifiers(UUID, int, String)
instead.- Parameters:
datasetKey
- dataset keyattempt
- attempt to runmessage
- with failed metrics and other info*
-
allowAbsentIndentifiers
Mark failed identifier stage as finished and continue interpretation process for datasets were identifier stage failed because of a threshold limit- Parameters:
datasetKey
- dataset keyattempt
- attempt to run
-
allowAbsentIndentifiers
Mark latest failed identifier stage as finished and continue interpretation process for datasets were identifier stage failed because of a threshold limit- Parameters:
datasetKey
- dataset key
-
notifyAbsentIdentifiers
Sends a notification to the data administrators about absent identifiers issues with the dataset.- Parameters:
datasetKey
- key of the datasetattempt
- crawling attemptexecutionKey
- key of the pipelines executionmessage
- cause of the issue
-