Skip to content

Added bulk object allocator (memory pool) for syntax nodes #173

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 20 commits into
base: master
Choose a base branch
from
Open

Added bulk object allocator (memory pool) for syntax nodes #173

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
20 commits
Select commit Hold shift + click to select a range
95233e9
Update README.md
vintagedave Aug 4, 2015
643d79f
Merge branch 'master' of https://github.com/vintagedave/DelphiAST
vintagedave Apr 1, 2016
e6baee2
Re-adding string caching (memory optimization, for memory usage and a…
vintagedave Apr 22, 2016
3228e5b
TValuedSyntaxNode uses the string cache
vintagedave Apr 22, 2016
66d852e
Lexer uses the string cache for tokens
vintagedave Apr 22, 2016
1aa71d6
TStringCacheDictionary is not used (was originally written for a dict…
vintagedave Apr 22, 2016
fc0470b
Basic threadsafety for the string cache. Not extensively reviewed. Wh…
vintagedave Apr 22, 2016
5b58370
Fixed typos for case where the ifdef to be threadsafe was turned off
vintagedave Apr 22, 2016
6685524
Comment; also holds lock for AddAndGet (enters lock individually for …
vintagedave Apr 22, 2016
38e35ba
Revert "Update README.md"
vintagedave Apr 22, 2016
a0d3bad
Syntax nodes can now optionally use a bulk allocator (a memory pool).…
vintagedave Apr 23, 2016
b799dd8
Information comment at the top of StringCache.pas
vintagedave Apr 23, 2016
85929d6
Added email address to info common at the top
vintagedave Apr 23, 2016
4dd3f18
Merge branch 'master' of https://github.com/RomanYankovsky/DelphiAST …
vintagedave Apr 27, 2016
72999f8
Merge branch 'roman-original-master' into syntaxnode-allocation
vintagedave Dec 30, 2018
a1c894a
Turn on syntax node allocation by default
vintagedave Dec 30, 2018
1cd072f
Updated readme to point at Roman's original, and add info about what'…
vintagedave Dec 30, 2018
9848973
Added Delphi syntax highlighting to the sample code snippet
vintagedave Dec 30, 2018
3a49933
Updated link, info on string interning being disabled
vintagedave Dec 30, 2018
42b93dd
Merge branch 'origin-master' of https://github.com/vintagedave/Delphi…
vintagedave Dec 30, 2018
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
401 changes: 13 additions & 388 deletions Demo/DelphiASTDemo.dproj
View file
Open in desktop

Large diffs are not rendered by default.

Binary file modified Demo/DelphiASTDemo.res
View file
Open in desktop
Binary file not shown.
2 changes: 1 addition & 1 deletion Demo/uMainForm.dfm
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -33,7 +33,7 @@ object MainForm: TMainForm
Width = 50
end>
end
object CheckBox1: TCheckBox
object chkStringIntern: TCheckBox
AlignWithMargins = True
Left = 3
Top = 350
Expand Down
6 changes: 3 additions & 3 deletions Demo/uMainForm.pas
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -14,8 +14,8 @@ TMainForm = class(TForm)
MainMenu: TMainMenu;
OpenDelphiUnit1: TMenuItem;
OpenDialog: TOpenDialog;
StatusBar: TStatusBar;
CheckBox1: TCheckBox;
StatusBar: TStatusBar;
chkStringIntern: TCheckBox;
procedure OpenDelphiUnit1Click(Sender: TObject);
private
procedure UpdateStatusBarText(const StatusText: string);
Expand Down Expand Up @@ -121,7 +121,7 @@ procedure TMainForm.OpenDelphiUnit1Click(Sender: TObject);
begin
if OpenDialog.Execute then
begin
OutputMemo.Lines.Text := Parse(OpenDialog.FileName, StatusText, CheckBox1.Checked);
OutputMemo.Lines.Text := Parse(OpenDialog.FileName, StatusText, chkStringIntern.Checked);
UpdateStatusBarText(StatusText);
end
end;
Expand Down
90 changes: 12 additions & 78 deletions README.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,89 +1,23 @@
### Abstract Syntax Tree Builder for Delphi
With DelphiAST you can take real Delphi code and get an abstract syntax tree. One unit at time and without a symbol table though.
### Abstract Syntax Tree Builder for Delphi
With DelphiAST you can take real Delphi code and get an abstract syntax tree.

FreePascal and Lazarus compatible.
### This fork

#### Sample input
```delphi
unit Unit1;

interface

uses
Unit2;

function Sum(A, B: Integer): Integer;
This is a fork of https://github.com/RomanYankovsky/DelphiAST which adds:

implementation
* Nodes are not repeatedly allocated and freed, instead using an object cache when a new one is created or destroyed (see `TSyntaxNode` in DelphiAST.Classes.pas. The interesting bit is:

function Sum(A, B: Integer): Integer;
begin
Result := A + B;
end;

end.
```delphi
class function NewInstance: TObject {$IFDEF AUTOREFCOUNT} unsafe {$ENDIF}; override;
procedure FreeInstance; override;
```

#### Sample outcome
```xml
<UNIT line="1" col="1" name="Unit1">
<INTERFACE begin_line="3" begin_col="1" end_line="10" end_col="1">
<USES begin_line="5" begin_col="1" end_line="8" end_col="1">
<UNIT line="6" col="3" name="Unit2"/>
</USES>
<METHOD begin_line="8" begin_col="1" end_line="10" end_col="1" kind="function" name="Sum">
<PARAMETERS line="8" col="13">
<PARAMETER line="8" col="14">
<NAME line="8" col="14" value="A"/>
<TYPE line="8" col="20" name="Integer"/>
</PARAMETER>
<PARAMETER line="8" col="17">
<NAME line="8" col="17" value="B"/>
<TYPE line="8" col="20" name="Integer"/>
</PARAMETER>
</PARAMETERS>
<RETURNTYPE line="8" col="30">
<TYPE line="8" col="30" name="Integer"/>
</RETURNTYPE>
</METHOD>
</INTERFACE>
<IMPLEMENTATION begin_line="10" begin_col="1" end_line="17" end_col="1">
<METHOD begin_line="12" begin_col="1" end_line="17" end_col="1" kind="function" name="Sum">
<PARAMETERS line="12" col="13">
<PARAMETER line="12" col="14">
<NAME line="12" col="14" value="A"/>
<TYPE line="12" col="20" name="Integer"/>
</PARAMETER>
<PARAMETER line="12" col="17">
<NAME line="12" col="17" value="B"/>
<TYPE line="12" col="20" name="Integer"/>
</PARAMETER>
</PARAMETERS>
<RETURNTYPE line="12" col="30">
<TYPE line="12" col="30" name="Integer"/>
</RETURNTYPE>
<STATEMENTS begin_line="13" begin_col="1" end_line="15" end_col="4">
<ASSIGN line="14" col="3">
<LHS line="14" col="3">
<IDENTIFIER line="14" col="3" name="Result"/>
</LHS>
<RHS line="14" col="13">
<EXPRESSION line="14" col="13">
<ADD line="14" col="15">
<IDENTIFIER line="14" col="13" name="A"/>
<IDENTIFIER line="14" col="17" name="B"/>
</ADD>
</EXPRESSION>
</RHS>
</ASSIGN>
</STATEMENTS>
</METHOD>
</IMPLEMENTATION>
</UNIT>
```
* String interning, using a different technique to the one currently in DelphiAST (written afterwards? The code in this fork dates from 2016.) This is disabled by default currently since the latest DelphiAST has its own technique.

The general idea is to try to prevent memory fragmentation or many allocations and de-allocations when DelphiAST is used constantly. This code is used in the [Parnassus Bookmarks and Navigator plugins](https://parnassus.co/delphi-tools/), which regularly parse the current unit when the user types. That can be many times an hour, even many times a minute. Early versions had users reporting the IDE gave out of memory errors where there was still a lot of free memory, a classic indication of fragmentation. Releases using this code, especially the string interning, solved those bug reports.

#### Copyright
Copyright (c) 2014-2017 Roman Yankovsky (roman@yankovsky.me) et al
Copyright (c) 2014-2017 Roman Yankovsky (roman@yankovsky.me) et al (these changes are copyright David Millington 2016-2018.)

DelphiAST is released under the Mozilla Public License, v. 2.0

Expand Down
Loading