Resource requirement min max validation

[Stellatsuu]

Related to : #2163

• Added resource requirement minmax validation
• Added validation when running --validate
• Added related tests

[mr-c]

Thank you @Stellatsuu !

[codecov]

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 84.92%. Comparing base (0cbbd92) to head (4937dbc).

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #2179      +/-   ##
==========================================
+ Coverage   84.85%   84.92%   +0.07%     
==========================================
  Files          46       46              
  Lines        8509     8523      +14     
  Branches     1984     1988       +4     
==========================================
+ Hits         7220     7238      +18     
- Misses        812      814       +2     
+ Partials      477      471       -6     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
• 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

[Stellatsuu]

cwltool --disable-validate tests/wf/bad_resreq_mnmx_wf.cwl command works while it shouldn't, since the workflow resource requirement is wrong:

#!/usr/bin/env cwl-runner
class: Workflow
cwlVersion: v1.2

requirements: <-----
  ResourceRequirement:
      ramMin: 128
      ramMax: 64

inputs: []
outputs: []

steps:
  hello_world:
    requirements:
      ResourceRequirement:
        ramMin: 64
        ramMax: 128
    run:
        class: CommandLineTool
        baseCommand: [ "echo", "Hello World" ]
        inputs: [ ]
        outputs: [ ]
    out: [ ]
    in: [ ]

I looked at the code, and it seems that the main reason is that, evalResources is only called when the cwl object is not a Workflow:

def Process(...):
   def _init_job(...):
        ...
        if self.tool["class"] != "Workflow":
            builder.resources = self.evalResources(builder, runtime_context)
        return builder
   def evalResources(...):
        -validation is here-

How should I manage to check if the resource requirements of a Workflow are correct when calling --disable-validate?

[Stellatsuu]

Hello @mr-c,
I hope you're doing well.
Could you, please, take a quick look at this pull request? Especially this part: #2179 (comment) where I'm not sure of what to do to make it work as intended

Thank you in advance

[mr-c]

Sorry for the delay, @Stellatsuu , I missed your original question; so thank you for the reminder

5f455a4 is where that variation was added, as part of implementing --parallel

[mr-c]

This doesn't work in the case of a CWL Expression or Parameter Reference (type str).

Either
A. Skip strings here
or
B. move the call to this function to later, when a builder object is available to utilize builder.do_eval to get the actual values.

[Stellatsuu]

Hello @mr-c,

I removed the cast here so that string values are skipped. This part of the code is only executed when running cwltool --validate. The actual runtime validation is performed later in builder.resources = self.evalResources(builder, runtime_context), so Expressions should work here.

My question is: if we want to --validate a requirement that contains a CWL expression, we first need to evaluate the expression to get an int or float before running the validation. But since evaluating expressions requires a Builder, how could this be done during validation? Validation does not use builder right?

[Stellatsuu]

Hello @mr-c, I hope you're doing well.

Could you please take a look at this comment? #2179 (comment)
If not being able to use --validate on a CWL Expression requirements is not an issue for you for now, then the PR should be ready to be reviewed and potentially merged.