Add new rule for mit

[alok1304]

Now we detects the correct license expression i.e mit.

Fixes #3860
Fixes #3861

Tasks

• Reviewed contribution guidelines
• PR is descriptively titled 📑 and links the original issue above 🔗
• Tests pass -- look for a green checkbox ✔️ a few minutes after opening your PR
  Run tests locally to check for errors.
• Commits are in uniquely-named feature branch and has no merge conflicts 📁
• Updated documentation pages (if applicable)
• Updated CHANGELOG.rst (if applicable)

Signed-off-by: Alok Kumar alokkumarjipura9973@gmail.com

[alok1304]

@pombredanne can you please review this PR, Now there is no bug. We detects correct mit license Expression.

Eg:

  "files": [
    {
      "path": "_context.py",
      "type": "file",
      "detected_license_expression": "mit",
      "detected_license_expression_spdx": "MIT",
      "license_detections": [
        {
          "license_expression": "mit",
          "license_expression_spdx": "MIT",
          "matches": [
            {
              "license_expression": "mit",
              "license_expression_spdx": "MIT",
              "from_file": "_context.py",
              "start_line": 2,
              "end_line": 2,
              "matcher": "2-aho",
              "score": 100.0,
              "matched_length": 10,
              "match_coverage": 100.0,
              "rule_relevance": 100,
              "rule_identifier": "mit_1351.RULE",
              "rule_url": "https://github.com/nexB/scancode-toolkit/tree/develop/src/licensedcode/data/rules/mit_1351.RULE",
              "matched_text": "# MIT License (see LICENSE or https://opensource.org/licenses/MIT)",
              "matched_text_diagnostics": "MIT License (see LICENSE or https://opensource.org/licenses/MIT)"
            }
          ],
          "identifier": "mit-cf1210d2-5820-2542-c850-cac164e24fed"
        }
      ],
      "license_clues": [],
      "percentage_of_license_text": 0.24,
      "scan_errors": []
    }
  ]

This new rule fixes two bugs that are facing in #3861 and #3860.

Hope this helps.

[alok1304]

@pombredanne can you please review this PR it solves two bugs.

[AyanSinhaMahapatra]

@alok1304 Thanks++
See comments for suggested improvements.
To fix license detection false positive issues by updating rules you need to:

1. Add new rules as applicable
2. Also add required phrases/minimum_coverage to the falsely detected rules with different licenses which are not present: For example here you also need to modify the src/licensedcode/data/rules/cc-by-3.0_and_mit_3.RULE RULE (see the scan result JSON you pasted for this), by adding required phrases to the license names + URLs seperately, so similar issues does not happen in the future with this rule. Please also apply the same to other/future PRs. Eventually we are doing this massively in Update rules with required phrases automatically #3924 to improve rules.

[pombredanne]

@alok1304 This overall look good! 👍
You can improve this further this way:

1. create tests adding a test and expected file in https://github.com/aboutcode-org/scancode-toolkit/tree/develop/tests/licensedcode/data/datadriven/lic4 ... see all examples of test file pairs there.

The test for #3860 and #3861 would be the same with this text (like for https://github.com/aboutcode-org/scancode-toolkit/blob/develop/tests/licensedcode/data/datadriven/lic4/2675-sqlite.cpp )

# Copyright: (c) 2020, Jordan Borean (@jborean93) <jborean93@gmail.com>
# MIT License (see LICENSE or https://opensource.org/licenses/MIT)

And expected YAML file, (like for https://github.com/aboutcode-org/scancode-toolkit/blob/develop/tests/licensedcode/data/datadriven/lic4/2675-sqlite.cpp.yml )

license_expressions:
  - mit

2. Also add a few new rules with this related contents (this can be a separate PR alright):

---
license_expression: mit
is_license_notice: yes
relevance: 100
referenced_filenames:
    - LICENSE
ignorable_urls:
    - https://opensource.org/licenses/MIT
---

{{MIT License (see LICENSE or https://opensource.org/licenses/MIT) }}

And another:

---
license_expression: mit
is_license_notice: yes
relevance: 100
referenced_filenames:
    - LICENSE
---

{{MIT License (see LICENSE) }}

And a few variations that can bee seen in the wild if we do not detect these exactly:

• Search for .MD variants: https://github.com/search?q="MIT+License+(see+LICENSE.md)"&type=code
  • MIT License (see LICENSE.md)
  • distributed under MIT license (see LICENSE.md). https://github.com/modeladonicho/Maxent/blob/a500e9108e7edeeea8558cd8ed43e622bedc410d/license-summary.txt#L1
  • https://github.com/zeux/meshoptimizer/blob/58b0ff55c04c650c873409c0f557e7810a1f1a35/README.md?plain=1#L598
  • https://github.com/search?q="MIT+License+(see+LICENSE.md)"&type=code
  • https://github.com/benderjs/benderjs/blob/39c2159d882d0837ddbfd7b8a19a2ce39738f75a/lib/constants.js#L3
  • https://github.com/inunix3/rxpipes/blob/d99c35f2d1d5f623f07c9b74356015480e557a1c/src/pipe.rs#L3
  • https://github.com/marcusschiesser/openbahn-api/blob/9f50b34293837bd0afe8e31c93e3911c06524b7e/README.md?plain=1#L14
  • https://github.com/GrenDrake/glulx-assemble/blob/213a452d186be4fbe9ad449f2de2249244e6b483/demos/minimal.ga#L7
  • https://github.com/iddev5/cobalt/blob/0de6c1600bad5213ec8d053e654300bf54a2b721/cobalt#L5
  • https://github.com/LAiSR-SK/AiRobustnessTestingKit-AiR-TK-/blob/39b0632e55bd3768267f8451c76f532cbe79ca3a/src/airtk/__init__.py#L2
  • https://github.com/Stevie-O/delayed-ldr-controlled-led/blob/fbbeb26ab969cdd0135812ea2c100e13adaa962b/ldrmon.h#L8
  • https://github.com/mlewand/benderjs-yui/blob/c7112ada1af11823cc4d5b89f12c666add027c07/lib/index.js#L3
  • https://github.com/jcoliz/defender-endpoint-samples/blob/3f32a2c7fa6ee9de7210aaace33710b1487da80d/hello-world/Program.cs#L2
  • https://github.com/anmarques/CLoVER/blob/71996b3cda3e91ecd9696d89d6a088215062e203/source/GP.py#L6
  • https://github.com/zeux/meshoptimizer/blob/58b0ff55c04c650c873409c0f557e7810a1f1a35/README.md?plain=1#L598

And all the variations where there is a LICENSE.txt:

• https://github.com/search?q="MIT+License+(see+LICENSE.txt)"&type=code

And a few rst:

• https://github.com/search?q="MIT+License+(see+LICENSE.rst)"&type=code

And more variants:

• https://github.com/search?q="MIT+License+(see+LICENSE+file)"&type=code