The architecture is based on the way modern code editors and IDEs are extended to support additional languages. We chose to implement Language Server Protocol (LSP), which is supported by a majority of contemporary editors.
In LSP, the two parties that communicate are called a client and a language server. A simple example is displayed in the picture above. The client runs as a part of an editor. The language server may be a standalone application that is connected to the client by a pipe or TCP. All language-specific user actions (for example the Go to definition command) are transformed into standard LSP messages and are sent to the language server. The language server then analyzes the source code and sends back a response, which is then interpreted and presented to the user in an editor-specific way. This architecture makes it possible to only have one LSP client implementation for each code editor, which can be reused by all programming languages. And vice versa, every language server can be easily used by any editor that has an implementation of the LSP client.
To add support for HLASM, we have implemented the LSP language server and written a lightweight extension to an editor, which uses an already existing implementation of the LSP client. To implement source code highlighting, we had to extend the protocol with a new notification. This notification is used for transferring information from the language server to the VS Code client, which is extended to highlight code in the editor based on the incoming custom notifications.
In this chapter, we further decompose the project into smaller components and describe their relationships. The two main components are the parser library and the language server — an executable application that uses the parser library. An overview of the architecture is in the picture below. The architecture of the whole project is shown in Architecture visualization.
Language Server ComponentThe responsibility of the language server component is to maintain the LSP session, convert incoming JSON messages and use the parser library to execute them. The functionality includes:
reading LSP messages from either a standard input or TCP and writing responses
parsing JSON RPC to C++ structures, so they can be further used
serializing C++ structures into JSON, so it can be sent back to the client
asynchronous request handling: when a user makes several consecutive changes to a source code, parsing on every change is not needed
Parser library is the core of the project — it encapsulates the analyzer, which provides all parsing capabilities, and workspace manager, which keeps track of open files in the editor and manages their dependencies. It has to keep the representation of workspaces and files in the parser library exactly the same as the user sees in the editor. It also starts the analyzer when needed, manages workspace configuration and provides external macro and copy libraries to analyzer.
The parser library API is based on LSP — every relevant request and notification has a corresponding method in the parsing library.
At first, the API implements the LSP notifications that ensure the editor state synchronization. Apart from working with individual files, the LSP also supports workspaces. A workspace is basically just a folder that contains related source codes. The LSP also supports working with multiple workspaces at the same time. We use it when searching for dependencies of HLASM source codes (macros, and copy files).
The parser library has the exact contents of all files in open workspaces. To achieve that, there is a file watcher running in the LSP client that notifies the server when any of the HLASM source files is changed outside of editor. For example, when a user deletes an external macro file, the parser library reacts by reporting that it cannot find the macro.
The list of necessary editor state synchronization notifications is as follows:
Text synchronization notifications (didOpen, didChange, didClose), which inform the library about files that are currently open in the editor and their exact contents.
DidChangeWorkspaceFolders, which informs the library when a workspace has been opened or closed.
DidChangeWatchedFiles
Next, the API implements the requests and notifications that provide the parsing results, specifically:
publishDiagnostics. A diagnostic is used to indicate a problem with source files, such as a compiler error or a warning. The parser library provides a callback to let the language server know that diagnostics have changed.
Callback for highlighting information provision.
Language feature requests (definition, references, hover, completion), which provide information needed for proper reaction of the editor on user actions.
The analyzer processes a single HLASM file. It takes the contents of a source file by common string and a callback that can parse external files with a specified name. It provides a list of diagnostics linked to the file, highlighting, list of symbol definitions, and more.
The analysis of HLASM code includes:
recognition of statements and their parts (lexing and parsing)
interpretation of instructions that are executed in compile time
reporting of problems with the source by producing LSP diagnostics
providing highlighting and LSP information
HLASM source files have dependencies — other files that define macros or files brought in by the COPY instruction. Dependencies are only discovered when files are processed, so it is not possible to provide the files with macro definitions beforehand. The analyzer gets a callback that finds a file with a specified name, parses its contents and returns it as a list of parsed statements.
Client-Side VS Code ExtensionThe VS Code extension component ensures seamless integration with the editor. Its functions are:
to start the HLASM language server and the LSP client that comes with VS Code, and to create a connection between them.
to implement an extension of the LSP protocol for enabling server-side highlighting. The extended client parses the information from the server and uses the VS Code API to color the text in the editor.
to implement support for editing lines with continuations — when the user types something in front of the continuation character, it remains in place.
The macro tracer allows you to trace the compilation of HLASM source code in a way similar to common debugging. This is why we chose to implement support for the Debug Adapter Protocol (DAP). It is very similar to LSP, so most of the code implementing LSP in the language server component can be reused for both protocols.
The language server component communicates with the macro tracer component in the parser library. Its API mirrors the requests and events of the DAP.
The main responsibilities of the macro tracer include:
launch, continue, next, stepIn and disconnect requests, which allow the user to control the flow of the compilation
SetBreakpoints, which transfers the information about breakpoints that the user has placed in the code
Threads, StackTrace, Scopes and Variables requests, which allow the DAP client to retrieve information about the current processing stack (stack of nested macros and copy instructions), available variable symbols and their values
stopped, exited and terminated events, which let the DAP client know about state of traced source code
The macro tracer communicates with the workspace manager to retrieve the content of the traced files. It analyzes the source file in a separate thread and receives callbacks from the analyzer before each statement is processed. In the callback, the tracer puts the thread to sleep and waits for user interaction. During this time, it is possible to retrieve all variable and stack information from the processing and display it to the user.
RetroSearch is an open source project built by @garambo | Open a GitHub Issue
Search and Browse the WWW like it's 1997 | Search results from DuckDuckGo
HTML:
3.2
| Encoding:
UTF-8
| Version:
0.7.4