v31.1.0

What's Changed

• Migrate npm importer by @TG1999 in #960
• Migrate retiredotnet importer by @TG1999 in #1041
• Link sanity by @Hritik14 in #1048
• Handle purl fragments in package search #1032 by @TG1999 in #1033
• Ingest npm data through github api #1025 by @TG1999 in #1027
• Prepare for release v31.1.0 by @TG1999 in #1062

Full Changelog: v31.0.0...v31.1.0

v31.0.0

This is a major new release with data changes that are API breaking: The way we store CVSS scores has changed.
There is a major new feature with Vulntotal which is like https://www.virustotal.com/ for comparing vulnerability databases. We also re-enabled PostgreSQL advisory imports.

What's Changed

• Add initial config for vulntotal by @keshav-space in #777
• Add support for calculating CVSS score from the CVSS vector by @ziadhany in #747
• Add Vulntotal CLI by @keshav-space in #801
• Add GitHubDataSource by @keshav-space in #804
• Add OSS-Index DataSource by @keshav-space in #829
• Add Gitlab datasource by @keshav-space in #883
• Register available datasources by @keshav-space in #901
• Add Vulntotal by @pombredanne in #1009
• Migrate postgresql.py by @johnmhoran in #985
• Fix the API key request form UI and make it consistent with rest of UI by @TG1999 in #1004
• Explicitly state app name in TestMigration by @JonoYang in #1012
• Make bulk search fast by @TG1999 in #1017

New Contributors

• @JonoYang made their first contribution in #1012

Full Changelog: v30.3.1...v31.0.0

v30.3.1

This is a minor bug fix release.

• We enabled proper CSRF configuration for deployments
• We improved the content of API key request emails

What's Changed

• Fix csrf by @pombredanne in #998

Full Changelog: v30.3.0...v30.3.1

v30.3.0

This is a feature update release including minor bug fixes and the introduction of API keys and API throttling.

What's Changed

• Enable throttling by @TG1999 in #988
• Override throttle rate for each endpoint by @TG1999 in #993
• Add API authentication, key request and documentation by @pombredanne in #987
• Improve NVD handling and more by @pombredanne in #997

Full Changelog: v30.2.1...v30.3.0

v30.2.0

This is a critical bug fix release including features updates.

• We fixed critical performance issues that made the web UI unusable. This include
  removing some less interesting redundant details displayed in the web UI for
  vulnerabilities.
• We made minor documentation updates.
• We re-enabled support for Arch linux, Debian, and Ubuntu security advisories importers
• We added a new improver for Oval data sources
• We improved Alpine linux and Gitlab security advisories importers

The summary of performance improvements include these fixes:

• Cascade queries from exact to approximate searches to avoid full table scans
  in all cases. This is a band-aid for now. The proper solution will likely
  require using full text search instead.
• Avoid iceberg queries with "prefetch related" to limit the number of queries
  that are needed in the UI
• Do not recreate querysets from scratch but instead allow these to be chained
  for simpler and correct code.
• Remove extra details from the vulnerability pacge: each package was further
  listing its related vulnerabilities creating an iceberg query.
• Enable the django-debug-toolbar with a setting to easily profile queries on demand
  by setting both VULNERABLECODE_DEBUG and VULNERABLECODE_DEBUG_TOOLBAR enviroment
  variables.

What's Changed

• Refactor Gitimporter using fetchcode by @ziadhany in #817
• test redhat importer performance by profiling by @ziadhany in #843
• Migrate archlinux importer by @johnmhoran in #935
• Fix gitlab importer by @TG1999 in #959
• Migrate debian-oval and ubuntu importer by @TG1999 in #740
• Make search for vulnerabilities faster by @pombredanne in #955
• Update RTD overview by @johnmhoran in #964
• Prepare release 30.2.0 by @pombredanne in #968

Full Changelog: v30.1.1...v30.2.0

v30.1.1

What's Changed

• Add API link/info to navbar by @johnmhoran in #948
• Prepare release 30.1.1 by @pombredanne in #951

Full Changelog: v30.1.0...v30.1.1

v30.1.0

What's Changed

• Add a PyPa importer, organize the code using a shared osv.py by @ziadhany in #780
• Merge Release 30.0.0 branch in main by @pombredanne in #934
• Add API endpoint to get all vulnerable packages #929 by @TG1999 in #932
• Prepare for v30.1.0 by @TG1999 in #938

Full Changelog: v30.0.0...v30.1.0

v30.0.0

Version v30.0.0

This is a major version that is not backward compatible.

• We refactored the core processing with Importers that import data and Improvers that
  transform imported data and convert that in Vulnerabilities and Packages. Improvers can
  also improve and refine imported and existing data as well as enrich data using external
  data sources. The migration to this new architecture is under way and not all importers
  are available.

  Because of these extensive changes, it is not possible to migrate existing imported
  data to the new schema. You will need instead to restart imports from an empty database
  or access the new public.vulnerablecode.io live instance. We also provide a database dump.

• You can track the progress of this refactoring in this issue:
  #597

• We added new data sources including PYSEC, GitHub and GitLab.

• We improved the documentation including adding development examples for importers and improvers.

• We removed the ability to edit relationships from the UI. The UI is now read-only.

• We replace the web UI with a brand new UI based on the same overall look and feel as ScanCode.io.

• We added support for NixOS as a Linux deployment target.

• The aliases of a vulnerabily are reported in the API vulnerabilities/ endpoint

• There are breaking Changes at API level with changes in the data structure:

  • in the /api/vulnerabilities/ endpoint:

    • Rename resolved_packages to fixed_packages
    • Rename unresolved_packages to affected_packages
    • Rename url to reference_url in the reference list
    • Add is_vulnerable property in fixed and affected_packages.

  • in the /api/packages/ endpoint:

    • Rename unresolved_vulnerabilities to affected_by_vulnerabilities
    • Rename resolved_vulnerabilities to fixing_vulnerabilities
    • Rename url to reference_url in the reference list
    • Add new attribute is_resolved
    • Add namespace filter

• We have provided backward compatibility for url and unresolved_vulnerabilities for now.
  These will be removed in the next major version and should be considered as deprecated.

• There is a new experimental cpe/ API endpoint to lookup for vulnerabilities by CPE and
  another aliases/ endpoint to lookup for vulnerabilities by aliases. These two endpoints will be
  replaced by query parameters on the main vulnerabilities/ endpoint when stabilized.

• Added filters for vulnerabilities endpoint to get fixed packages in accordance
  to the details given in filters: For example, when you call the endpoint this way
  /api/vulnerabilities?type=pypi&namespace=foo&name=bar, you will receive only
  fixed versioned purls of the type pypi, namespace foo and name bar.

• Package endpoint will give fixed packages of only those that
  matches type, name, namespace, subpath and qualifiers of the package queried.

• Paginated initial listings to display a small number of records
  and provided page per size with a maximum limit of 100 records per page.

• Add fixed packages in vulnerabilities details in packages endpoint.

• Add bulk search support for CPEs.

• Add authentication for REST API endpoint.
  The autentication is disabled by default and can be enabled using the
  VULNERABLECODEIO_REQUIRE_AUTHENTICATION settings.
  When enabled, users have to authenticate using
  their API Key in the REST API.
  Users can be created using the Django "createsuperuser" management command.

• The data license is now CC-BY-SA-4.0 as this is the highest common
  denominator license among all the data sources we collect and aggregate.

Other:

• We dropped calver to use a plain semver.
• We adopted vers and the new univers library to handle version ranges.

What's Changed

• Improve error handling and other misc. updates by @pombredanne in #267
• Fixed the spelling mistakes and grammatical errors by @Abhigyankrsingh in #269
• Add Apache HTTPD advisory importer by @sbs2001 in #261
• Add kaybee statement importer by @sbs2001 in #263
• Use packageurl version 0.9.3 and Add nginx importer by @sbs2001 in #264
• Add postgresql importer by @sbs2001 in #265
• Use skeleton project structure by @sbs2001 in #274
• Added faq section by @tushar912 in #283
• Update docs by @sbs2001 in #271
• Adapt rust importer to new advisory format by @sbs2001 in #281
• Use GH action instead travis for CI. by @sbs2001 in #295
• Add SOURCES.rst to document data sources being used by @sbs2001 in #298
• Cleanup codebase and fix minor bugs and other improvements by @sbs2001 in #278
• Import apache tomcat by @sbs2001 in #292
• Improve GitHub importer by @sbs2001 in #291
• Stop debian importer from collecting temp vulnerabilities by @sbs2001 in #285
• Add tests for nginx and postgres importers by @sbs2001 in #301
• Elixir Security Importer by @tushar912 in #294
• Bump lxml from 4.3.3 to 4.6.2 by @dependabot in #306
• Verbose name plural for 'PackageRelatedVulnerability' by @Shivam-316 in #309
• Use drf-spectacular instead of drf-yasg for API docs by @sbs2001 in #310
• Add endpoints for bulk requesting vulnerabilities and packages by @sbs2001 in #303
• Don't allow null values for qualifiers by @sbs2001 in #313
• Add nix support by @rolfschr in #275
• Fix package result count in web ui by @sbs2001 in #329
• Collect references from github importer by @sbs2001 in #331
• Add django admin functionality for searching and filtering objects by @sbs2001 in #330
• Add message when no vulnerabilities are found for a vuln_id by @tushar912 in #337
• Change Alpine data source to use new source by @sbs2001 in #339
• Store severity scores by @sbs2001 in #290
• Improve UI by @sbs2001 in #335
• Fix regex in schema validator in alpine importer by @sbs2001 in #347
• Improve docs by @pombredanne in #316
• Collect kafka cves by @sbs2001 in #342
• Make trailing slash optional in apis by @sbs2001 in #350
• Update Nix deps to incorporate latest Python packages by @rolfschr in #352
• Disable schema validation for alpine linux to fix nix test by @sbs2001 in #353
• Collect suse scores by @sbs2001 in #354
• Collect archlinux severity scores by @sbs2001 in #355
• Handle vulnerabilities which don't have any vulnerability ids by @sbs2001 in #259
• Collect ghsa severity by @sbs2001 in #358
• Use case insensitive inexact lookups for search views by @sbs2001 in #360
• Make RedHat CVE import more robust by @pombredanne in #319
• Refactor codebase and tests to treat Advisory class mutable by @sbs2001 in #363
• Improve Ubuntu OVAL importer by @pombredanne in #322
• Bump aiohttp from 3.6.2 to 3.7.4 by @dependabot in #364
• Update nix deps by @rolfschr in #367
• UI compress vuln view by @sbs2001 in #368
• Update pypi deps db to 2021-03-06. by @rolfschr in #370
• Update README.rst by @InLaw in #371
• Send severity data along with vulnerability in bulk api by @sbs2001 in #369
• Use a more specific url for cvss qualitative severity system. by @tushar912 in #373
• Add istio importer and tests by @tushar912 in #336
• [Refactor] Rename vuln_references to references by @imnitishng in #377
• Explicity provide lxml parser to beautifulsoup by @Hritik14 in #382
• Correct API docs path and fix pytest invocation by @Hritik14 in #379
• Sanity Checks for redhat import response by @savish28 in #387
• Make sure vulnerability id is_cve or is_vulcoid by @Hritik14 in https:...

v30.0.0rc6

What's Changed

• Prepare v30.rc6 by @pombredanne in #914

Full Changelog: v30.0.0rc5...v30.0.0rc6

v30.0.0rc5

What's Changed

• Implement initial set of RTD updates #885 #886 #887 #888 by @johnmhoran in #890
• Allow case insensitive search for VCIDs #875 by @TG1999 in #898
• Make URLs mandatory for references #891 by @TG1999 in #899
• Improve UI by @pombredanne in #894
• Migrate from VULCOID to VCID #811 by @TG1999 in #896
• Prepare release 30 by @pombredanne in #909

Full Changelog: v30.0.0rc4...v30.0.0rc5