Otp 27 #9

KornelH · 2024-04-02T09:20:00Z

Erlang/OTP 27 introduces two new language elements, triple-quoted strings (docstrings) and sigils (type prefix for string literals).

Two simple example modules to test the syntax highlight with the original and updated TextMate language definition:

-module(docstring_examples).

-compile(export_all).

-doc """
     Docstring examples
     """.
f() ->
    """
  Line "1"
  Line "2"
  """ = "Line \"1\"
Line \"2\"",
    X = lists:seq(1,3), % just to check is syntax highlight is still ok

    """
      First line starting with two spaces
    Not escaped: "\t \r \xFF" and """

    """ = "  First line starting with two spaces
Not escaped: \"\\t \\r \\xFF\" and \"\"\"
",
    X = lists:seq(1,3), % just to check is syntax highlight is still ok

    """""
""""
FIXME: previous line is not a closing delimiter because opening-closing delimiter have five double quote characters
""""" = "\"\"\"\"",
    X = lists:seq(1,3), % just to check is syntax highlight is still ok

    ok.

-define(THIS_IS_THE_END, "end"). % just to check is syntax highlight is still ok

-module(sigil_examples).

-compile(export_all).

-doc """
     Sigil examples
     """.
-spec f() -> ok.
f() ->
     ~(monkey ~2..0b\n),
    ~b(monkey ~2..0b\n),
    ~B(monkey ~2..0b\n), % verbatim
    ~s(monkey ~2..0b\n),
    ~S(monkey ~2..0b\n), % verbatim

     ~{monkey ~2..0b\n},
    ~b{monkey ~2..0b\n},
    ~B{monkey ~2..0b\n}, % verbatim
    ~s{monkey ~2..0b\n},
    ~S{monkey ~2..0b\n}, % verbatim

     ~[monkey ~2..0b\n],
    ~b[monkey ~2..0b\n],
    ~B[monkey ~2..0b\n], % verbatim
    ~s[monkey ~2..0b\n],
    ~S[monkey ~2..0b\n], % verbatim

     ~<monkey ~2..0b\n>,
    ~b<monkey ~2..0b\n>,
    ~B<monkey ~2..0b\n>, % verbatim
    ~s<monkey ~2..0b\n>,
    ~S<monkey ~2..0b\n>, % verbatim

     ~/monkey ~2..0b\n/,
    ~b/monkey ~2..0b\n/,
    ~B/monkey ~2..0b\n/, % verbatim
    ~s/monkey ~2..0b\n/,
    ~S/monkey ~2..0b\n/, % verbatim

     ~|monkey ~2..0b\n|,
    ~b|monkey ~2..0b\n|,
    ~B|monkey ~2..0b\n|, % verbatim
    ~s|monkey ~2..0b\n|,
    ~S|monkey ~2..0b\n|, % verbatim

     ~'monkey ~2..0b\n',
    ~b'monkey ~2..0b\n',
    ~B'monkey ~2..0b\n', % verbatim
    ~s'monkey ~2..0b\n',
    ~S'monkey ~2..0b\n', % verbatim

     ~"monkey ~2..0b\n",
    ~b"monkey ~2..0b\n",
    ~B"monkey ~2..0b\n", % verbatim
    ~s"monkey ~2..0b\n",
    ~S"monkey ~2..0b\n", % verbatim

     ~`monkey ~2..0b\n`,
    ~b`monkey ~2..0b\n`,
    ~B`monkey ~2..0b\n`, % verbatim
    ~s`monkey ~2..0b\n`,
    ~S`monkey ~2..0b\n`, % verbatim

     ~#monkey ~2..0b\n#,
    ~b#monkey ~2..0b\n#,
    ~B#monkey ~2..0b\n#, % verbatim
    ~s#monkey ~2..0b\n#,
    ~S#monkey ~2..0b\n#, % verbatim

    X = lists:seq(1,3), % just to check is syntax highlight is still ok
    ok.

-spec g() -> ok. % just to check is syntax highlight is still ok
g() ->
    ~"""
    monkey ~2..0b\n
    business
    """,
    ~b""""
    monkey ~2..0b\n
    """
    business
    """",
    ~B"""""
    monkey ~2..0b\n
    """
    business
    """"",
    ~s""""""
    monkey ~2..0b\n
    """
    business
    """""",
    ~S"""""""
    monkey ~2..0b\n
    """
    business
    """"""",
    X = lists:seq(1,3), % just to check is syntax highlight is still ok

    <<"\"\\µA\""/utf8>> = <<$",$\\,194,181,$A,$">> =
        ~b"""
          "\\µA"
          """ = ~b'"\\µA"' =
        ~B"""
          "\µA"
          """ = ~B<"\µA"> =
        ~"""
          "\µA"
          """ = ~"\"\\µA\"" = ~/"\\µA"/
    X = lists:seq(1,3), % just to check is syntax highlight is still ok

    [$",$\\,$µ,$A,$"] =
        ~s"""
          "\\µA"
          """ = ~s"\"\\µA\"" = ~s["\\µA"] =
        ~S"""
          "\µA"
          """ = ~S("\µA") =
        """
        "\µA"
        """ = "\"\\µA\""
    X = lists:seq(1,3), % just to check is syntax highlight is still ok

    ok.

-define(THIS_IS_THE_END, "end").

robertoaloi · 2024-04-02T10:04:11Z

Thanks! Not sure if there's a way to have programmatic tests for the grammar. Either way, would it make sense to commit your examples in a dedicated folder, even if only for visual inspection?

robertoaloi · 2024-04-02T10:19:17Z

Erlang.plist

+				</dict>
+			</dict>
+			<key>comment</key>
+			<string>Only whitespace characters are allowed after the beggining and before the closing sequences and those cannot be in the same line</string>


Suggested change

<string>Only whitespace characters are allowed after the beggining and before the closing sequences and those cannot be in the same line</string>

<string>Only whitespace characters are allowed after the beginning and before the closing sequences and those cannot be in the same line</string>

robertoaloi · 2024-04-02T10:28:07Z

Before/after, for reference.

Before:

After:

robertoaloi · 2024-04-02T10:34:33Z

@KornelH This is an amazing and very welcome contribution! Let me know when you're ready, so I can merge this. I also like the idea of splitting this into multiple PRs (like you already did for the io:format parts). 🚀

KornelH · 2024-04-02T22:10:55Z

Unfortunately, I neither know any automatic way to test syntax highlight. Therefore, I'll add the example files for visual inspection similar to pull request #10. Also, I will break this commit into two separate commits about docstrings and sigils. I don't think these will change in the final version of OTP 27, so I will do it in 1-2 days.

garazdawi · 2024-04-03T09:00:11Z

As it happens I was also looking into providing the changes in this PR and started looking at how to do CI for the grammar. I've opened #11 with a first draft how we can run tests.

garazdawi · 2024-04-03T09:01:40Z

Another things I was looking at was seeing if we can provide an included grammar for docstrings so that they can be highlighted using the Markdown highlighter?

robertoaloi · 2024-04-03T14:33:27Z

Another things I was looking at was seeing if we can provide an included grammar for docstrings so that they can be highlighted using the Markdown highlighter?

That would be awesome!

KornelH · 2024-04-03T15:14:53Z

A small note, as far as I understand it, only -moduledoc and -doc have Markdown as default format, not all triple-quoted strings.
https://erlang.org/documentation/doc-15.0-rc2/doc/system/documentation.html

garazdawi · 2024-04-03T16:02:28Z

Any literal string (normal ”, triple quoted ””” or sigil ~”””) in a doc or moduledoc attribute is by default markdown.

KornelH · 2024-04-03T21:26:36Z

That's true. However in simple strings ("...") all special characters (even ") has to be escaped. Embedding Markdown into simple strings (of -doc or -moduledoc) with escaping all special characters feels a bit complex. In verbatim -doc or -moduledoc strings (e.g. ~S"..." or triple-quoted strings) I guess it's doable and useful.

Erlang/OTP 27 [1] introduces triple-quoted strings (docstrings) [2]. [1] https://www.erlang.org/news/168 [2] https://www.erlang.org/docs/27/system/data_types.html#string

Erlang/OTP 27 [1] introduces sigils (type prefix for string literals) [2]. [1] https://www.erlang.org/news/168 [2] https://www.erlang.org/docs/27/system/data_types.html#sigil

robertoaloi · 2024-04-04T07:51:09Z

@KornelH I noticed you opened this for review. Is that intentional or do you still want to break it into separate contributions for sigils and quoted strings? I am happy either way.

KornelH · 2024-04-04T08:16:43Z

Well, I can split into two requests.

KornelH · 2024-04-04T10:31:47Z

This pull request has been split into pull request #13 and #14.

KornelH · 2024-04-08T19:49:30Z

@garazdawi, I was playing a bit with embedding Markdown into -doc and -moduledoc attributes but the result feels very awkward. Personally, I feel it just disturbs me a lot and totally steals the focus from the source code. The below example is from OTP, lists:enumerate/1. Originally, I thought it's a good idea to embed Markdown but now I don't think that anymore. 🙁

garazdawi · 2024-04-09T06:50:59Z

hmm, yeah that was not very nice. Pretty neat that it highlights erlang examples in the markdown though :)

Do you know if it possible to configure the markdown style individually? Would you mind opening a PR (or just pointing to a branch) with what you have so that I can have a look at how it looks?

KornelH · 2024-04-09T07:10:00Z

I'm just learning it too and I have no idea if it's possible e.g. set a custom background or shade to embedded languages so it visually would look like a "comment blob" just like eDoc comments, and so it doesn't steal your attention unless you really want to read the documentation. I'll share my grammar changes soon.

garazdawi · 2024-04-09T07:27:13Z

It could also be the case this it becomes better if you indent the docs with 4 spaces. That way you get a visual queue that it is something different.

KornelH · 2024-04-10T20:23:44Z

@garazdawi, Please find my change in here: KornelH@309ded8

My problem with "indent the docs with 4 spaces" is that, most of the code that I read is not written by me and we cannot enforce the world how to write documentation comments. So, I think most of those will be just like in OTP as shown above. 🫤

garazdawi · 2024-04-11T07:23:35Z

My problem with "indent the docs with 4 spaces" is that, most of the code that I read is not written by me and we cannot enforce the world how to write documentation comments. So, I think most of those will be just like in OTP as shown above. 🫤

Yeah, I was mostly wondering if that fixed the issue. I'll have a look at what you have done and see if I can come up with any clever ideas :)

robertoaloi · 2024-04-11T07:24:07Z

I wonder how do other languages (e.g. Elixir) tackle this. Any approach which we can take inspiration from? Would it make sense to use the concept of a "scope" that could then be styled via themes?

robertoaloi · 2024-04-11T07:27:07Z

There's also something called embedded language. Maybe we could leverage that one?

garazdawi · 2024-04-11T08:05:36Z

While digging about I found this comment: microsoft/vscode#169956 (comment)

Seems like the way rust/js/ts does it is by having Markdown highlighting as a separate VS Code extension. That way the user can decide if they want it or not and possibly configure how it should look as well.

There's also something called embedded language. Maybe we could leverage that one?

This is what @KornelH has done in the prototype.

robertoaloi reviewed Apr 2, 2024

View reviewed changes

KornelH force-pushed the otp_27 branch from c68e284 to cd5739d Compare April 3, 2024 07:59

KornelH marked this pull request as ready for review April 3, 2024 08:04

robertoaloi mentioned this pull request Apr 3, 2024

Add sample for format control sequences #10

Merged

KornelH force-pushed the otp_27 branch from cd5739d to bae9f34 Compare April 3, 2024 21:18

Kornel Horvath added 2 commits April 4, 2024 07:34

Add docstrings as per Erlang/OTP 27

95d4395

Erlang/OTP 27 [1] introduces triple-quoted strings (docstrings) [2]. [1] https://www.erlang.org/news/168 [2] https://www.erlang.org/docs/27/system/data_types.html#string

Add sigils as per Erlang/OTP 27

65a3e67

Erlang/OTP 27 [1] introduces sigils (type prefix for string literals) [2]. [1] https://www.erlang.org/news/168 [2] https://www.erlang.org/docs/27/system/data_types.html#sigil

KornelH force-pushed the otp_27 branch from bae9f34 to 65a3e67 Compare April 4, 2024 05:35

KornelH closed this Apr 4, 2024

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Otp 27 #9

Otp 27 #9

KornelH commented Apr 2, 2024

robertoaloi commented Apr 2, 2024

robertoaloi Apr 2, 2024

robertoaloi commented Apr 2, 2024

robertoaloi commented Apr 2, 2024

KornelH commented Apr 2, 2024 •

edited

Loading

garazdawi commented Apr 3, 2024

garazdawi commented Apr 3, 2024

robertoaloi commented Apr 3, 2024

KornelH commented Apr 3, 2024

garazdawi commented Apr 3, 2024

KornelH commented Apr 3, 2024 •

edited

Loading

robertoaloi commented Apr 4, 2024

KornelH commented Apr 4, 2024 •

edited

Loading

KornelH commented Apr 4, 2024

KornelH commented Apr 8, 2024

garazdawi commented Apr 9, 2024

KornelH commented Apr 9, 2024

garazdawi commented Apr 9, 2024

KornelH commented Apr 10, 2024

garazdawi commented Apr 11, 2024

robertoaloi commented Apr 11, 2024

robertoaloi commented Apr 11, 2024

garazdawi commented Apr 11, 2024

	<string>Only whitespace characters are allowed after the beggining and before the closing sequences and those cannot be in the same line</string>
	<string>Only whitespace characters are allowed after the beginning and before the closing sequences and those cannot be in the same line</string>

Otp 27 #9

Otp 27 #9

Conversation

KornelH commented Apr 2, 2024

robertoaloi commented Apr 2, 2024

robertoaloi Apr 2, 2024

Choose a reason for hiding this comment

robertoaloi commented Apr 2, 2024

robertoaloi commented Apr 2, 2024

KornelH commented Apr 2, 2024 • edited Loading

garazdawi commented Apr 3, 2024

garazdawi commented Apr 3, 2024

robertoaloi commented Apr 3, 2024

KornelH commented Apr 3, 2024

garazdawi commented Apr 3, 2024

KornelH commented Apr 3, 2024 • edited Loading

robertoaloi commented Apr 4, 2024

KornelH commented Apr 4, 2024 • edited Loading

KornelH commented Apr 4, 2024

KornelH commented Apr 8, 2024

garazdawi commented Apr 9, 2024

KornelH commented Apr 9, 2024

garazdawi commented Apr 9, 2024

KornelH commented Apr 10, 2024

garazdawi commented Apr 11, 2024

robertoaloi commented Apr 11, 2024

robertoaloi commented Apr 11, 2024

garazdawi commented Apr 11, 2024

KornelH commented Apr 2, 2024 •

edited

Loading

KornelH commented Apr 3, 2024 •

edited

Loading

KornelH commented Apr 4, 2024 •

edited

Loading