Wednesday, October 16, 2024

Clustering High-Dimensional Data: Voice Separation in the Cocktail Party Effect

 Clustering is one of the most powerful techniques in machine learning, enabling us to group similar data points based on their features. When dealing with high-dimensional data—where each data point has a large number of features—traditional clustering methods face challenges due to the curse of dimensionality. However, powerful algorithms and techniques can still uncover hidden patterns, including voice separation in a noisy environment like the Cocktail Party Effect.

This blog will explore clustering in high-dimensional spaces and use voice separation as an example to show how these techniques can be applied in the real world.


What is High-Dimensional Data?

High-dimensional data refers to data that has a large number of features or variables. For example:

  • In an audio signal, each sample is described by multiple features, including frequency, amplitude, and time.
  • In an image, each pixel can be a feature, and there could be thousands or even millions of pixels.

As the number of features (dimensions) increases, several issues arise:

  • Curse of dimensionality: In higher dimensions, data points become more sparse, making it difficult to measure distances effectively.
  • Computational complexity: The more dimensions, the harder it becomes to compute pairwise distances or similarities.
  • Overfitting: More dimensions can introduce noise, leading to models that memorize the training data instead of generalizing to new data.

Clustering in High Dimensions

Clustering is a technique used to group data points that are similar based on their features. Common clustering algorithms like k-means or hierarchical clustering can work in high-dimensional spaces, but require adaptations or specialized techniques to handle the challenges of high dimensionality.

Challenges of Clustering High-Dimensional Data

  1. Distance metrics lose meaning: In higher dimensions, the difference in distance between points diminishes, which can reduce the effectiveness of algorithms based on distances like k-means.
  2. Sparsity: Data points in high dimensions are often sparse, making it difficult to define clear clusters.

To tackle these issues, advanced algorithms and dimensionality reduction techniques are often used to cluster high-dimensional data.

Popular Techniques for High-Dimensional Clustering

  • Dimensionality reduction: Methods like Principal Component Analysis (PCA) or t-SNE can be applied to reduce the number of dimensions while retaining the structure of the data.
  • Spectral clustering: Instead of directly clustering in high dimensions, spectral clustering transforms the data into a lower-dimensional space using graph theory, making the clusters more distinct.

These techniques enable machine learning models to extract meaningful information from high-dimensional data, even when there are hundreds or thousands of features.


The Cocktail Party Effect: Voice Separation as an Example

The Cocktail Party Effect refers to the human ability to focus on a single speaker’s voice in a noisy room full of other voices. Imagine being at a party with dozens of people speaking simultaneously. Your brain can zero in on one voice and tune out the others, much like clustering can help isolate meaningful signals from noisy data.

Voice Separation Problem

In machine learning, we can model the Cocktail Party Effect as a problem of separating multiple audio signals (voices) that are mixed together. This can be seen as a clustering problem in the frequency domain, where the goal is to cluster different sound sources (voices) and separate them.

How It Works: Using Machine Learning for Voice Separation

  1. Representing the Data (Feature Extraction):

    • Each voice can be represented as a high-dimensional audio signal, where each dimension corresponds to the frequency components over time.
    • Using techniques like Short-Time Fourier Transform (STFT), we can convert the raw audio waveform into a frequency-time representation, resulting in a high-dimensional matrix, where each element represents the energy at a certain frequency and time.
  2. Dimensionality Reduction:

    • Voice signals in a noisy environment have thousands of features (frequencies over time). To make clustering feasible, dimensionality reduction techniques like Non-negative Matrix Factorization (NMF) can be applied. NMF is particularly useful for separating mixed audio signals by finding a low-dimensional representation of the high-dimensional data.

    • In NMF, we attempt to approximate the input matrix XX (the high-dimensional frequency data) as a product of two lower-dimensional matrices:

    • Where:

      • X is the original high-dimensional matrix.
      • W represents the basis vectors (characteristics of individual voices).
      • H represents the encoding of these basis vectors over time.
  3. Clustering and Source Separation:

    • After dimensionality reduction, clustering techniques are applied to group together different components of the audio signal that belong to distinct voices.
    • The separated clusters correspond to different sound sources. Each cluster represents a specific voice, enabling the system to isolate one voice from the noisy background.
  4. Reconstructing the Voices:

    • Once the clusters (voices) have been identified, the system can reconstruct each individual voice by transforming the clustered frequency components back into the time domain using an inverse Fourier Transform. This process effectively isolates the voice of interest from the background noise.

Clustering Techniques in the Voice Separation Problem

Here are some clustering techniques that can be used for separating voices in the Cocktail Party Effect:

  • k-means Clustering: Although k-means is often less effective in very high-dimensional spaces, after dimensionality reduction, it can be used to cluster the audio data points based on frequency patterns.

  • Spectral Clustering: This technique is useful for clustering in cases where the geometry of the data is complex, as it works by using eigenvectors to find clusters in a transformed space.

  • Independent Component Analysis (ICA): ICA is often used in voice separation tasks because it assumes that the mixed signals (the cocktail of voices) are statistically independent. It separates out independent components (voices) from a mixed signal by maximizing statistical independence.


Conclusion: The Power of Clustering in High Dimensions

The problem of separating voices in a noisy environment like the Cocktail Party Effect is a perfect example of how clustering techniques can be applied to high-dimensional data. By reducing the dimensionality of complex audio signals and applying clustering algorithms, machine learning models can isolate individual voices from a mix of sounds.

This approach not only highlights the power of clustering in high-dimensional spaces but also shows how machine learning can tackle complex real-world problems like voice separation.

Through the combination of dimensionality reduction, clustering, and advanced machine learning algorithms, we can effectively separate signals (or voices) from noise, demonstrating the remarkable capabilities of machine learning in processing high-dimensional data.



Tuesday, October 15, 2024

The Cocktail Party Effect: Isolating a Single Voice Amid Noise Using Machine Learning

 Imagine being at a busy party, surrounded by conversations, music, and clinking glasses. Despite the overwhelming noise, you can still focus on a single conversation—the voice of your friend standing in front of you. This phenomenon, known as the Cocktail Party Effect, is a remarkable ability of the human brain to isolate and focus on a single sound source amid a noisy environment.

But how do machines mimic this ability? In this post, we'll explore the cocktail party effect and how modern machine learning techniques allow us to isolate a single voice in a crowded room, much like the brain does. We'll dive into voice separation, speaker recognition, and the machine learning algorithms that make it possible.


What is the Cocktail Party Effect?

The cocktail party effect refers to the human brain's ability to selectively focus on one sound, such as a single voice, while filtering out all the surrounding noise. It's a marvel of auditory processing that helps us navigate noisy environments. The brain leverages spatial cues, such as the location of the sound, the direction from which it comes, and the distinct characteristics of each voice, to make this possible.

For machines, recreating this capability involves complex algorithms and techniques that simulate the brain's selective hearing. While early attempts at speech separation relied on basic filtering methods, modern machine learning and deep learning approaches have revolutionized the process, making it more effective and scalable.


Challenges of Voice Separation

Isolating a single voice from a noisy environment presents several challenges:

  1. Overlapping Voices: When multiple people are speaking at once, their voices may overlap, making it difficult to differentiate between them.
  2. Background Noise: Sounds like music, traffic, or crowd noise can interfere with the clarity of the voice that needs to be isolated.
  3. Speech Variability: Different accents, speaking styles, and tones of voice add further complexity.
  4. Time Variability: Voices may overlap in time, making it harder to distinguish each speaker's turn.

These factors complicate the task of identifying and separating speech in real-world environments.


How Machine Learning Solves Voice Separation

Machine learning models for voice separation aim to address these challenges by recognizing the unique characteristics of individual speakers and filtering out background noise or other voices. Let's explore how this works.

1. Speech Separation Models

In the context of machine learning, speech separation refers to the process of isolating one or more voices from a mixture of sounds. This is typically achieved using neural networks, which are trained to recognize different voices based on features such as tone, pitch, and timbre.

Popular techniques include:

  • Deep clustering: This approach uses neural networks to group different sound sources into clusters based on their similarity. It works by embedding each sound source into a high-dimensional space where voices are grouped together, allowing separation.

  • Conv-TasNet (Convolutional Time-Domain Audio Separation Network): A neural network that operates directly on the raw audio waveform rather than its spectral representation, Conv-TasNet has proven highly effective in separating speech, even in overlapping conditions.

2. Speaker Diarization and Recognition

Speaker diarization refers to the process of identifying who is speaking and when. It’s often used in systems where multiple people are speaking in a conversation. Machine learning models can be trained to analyze audio input, segmenting it by the voice of each individual speaker.

  • Voiceprints: Just as fingerprints are unique to individuals, voiceprints capture the distinct features of each person’s voice. Machine learning algorithms learn to differentiate speakers by comparing these unique voiceprints in a process known as speaker recognition.

3. Source Separation Techniques

Source separation algorithms help machines extract a target speaker's voice from a mixture of sounds. These techniques often involve deep learning models like U-Net or Wave-U-Net, which learn to filter out the background noise and separate out individual sound sources.

  • Spectral masking: Spectral masking is a common method used in conjunction with deep learning. The machine model is trained to create a mask that highlights the desired speech frequency and attenuates others.

  • Recurrent Neural Networks (RNNs): RNNs can process sequences of audio data, making them suitable for speech tasks that involve multiple time steps. These networks "remember" information over time, making them effective for identifying and isolating individual voices from overlapping speech.


Practical Applications of Voice Separation

Isolating individual voices is an essential task in many real-world applications:

  1. Assistive Devices: For people with hearing impairments, devices that leverage machine learning for voice separation can significantly enhance their ability to focus on specific conversations in noisy environments.

  2. Speech Recognition Systems: Virtual assistants like Siri and Alexa rely on voice separation to process voice commands accurately, even in noisy rooms.

  3. Transcription Services: In environments like business meetings or courtrooms, separating speakers' voices allows accurate transcription of who said what.

  4. Surveillance: Security systems can use voice separation to isolate and analyze specific conversations in crowded public spaces.

  5. Media Production: Audio engineers in the music and film industries often use speech separation techniques to clean up recordings, isolating dialogue from background noise.


Conclusion: The Future of Speech Separation

As machine learning and artificial intelligence continue to advance, the ability of machines to replicate the cocktail party effect will become even more refined. New models are constantly being developed, pushing the boundaries of voice separation to improve accuracy in real-world scenarios.

Whether it's enhancing our daily interactions with voice-activated assistants or improving communication devices, voice separation is set to play a pivotal role in how we interact with machines in noisy environments. Machine learning is not just catching up to human abilities—it’s helping us reach new heights in sound processing.

By harnessing the power of deep learning and neural networks, the cocktail party effect can be replicated with impressive accuracy, allowing machines to focus on individual voices just like we do.


Key Takeaways:

  • The cocktail party effect is the brain’s ability to focus on a single voice in noisy environments.
  • Machine learning mimics this process using speech separation and speaker recognition.
  • Techniques like deep clustering, Conv-TasNet, and spectral masking are widely used.
  • Applications of voice separation span industries such as assistive devices, transcription services, and security.

As technology evolves, we can expect even more sophisticated approaches to solving the problem of voice separation, making our interactions with machines seamless and more intuitive.

Sunday, September 22, 2024

Custom Script Editor Project: Adding More Features (4th Enhancement)

 In this post, we continue with the fourth enhancement to the Custom Script Editor Project, adding exciting new features that bring more interactivity and dynamic capabilities to our script interpreter. These recent updates focus on enhancing the way data is fetched and displayed, and enabling the execution of program functions directly from within the script.

And as usual, if have not read the previous posts, here are the link: Custom Script Editor Project.

Here is the screenshot with the working enhancements:



1. Fetching Table Contents from the Server

As discussed in the previous post, we integrated a Flask Application to serve as our backend. This allows the script editor to fetch data dynamically from a server. The tables available on the server can now be accessed by the script editor, giving us real-time data capabilities. 

2. New Script Commands: Displaying Table Data and Structure

The newly introduced script command Table(showdata | showstructure) allows users to fetch and display both table data and table structure. This command can be written in the custom script, and depending on the argument (showdata or showstructure), the interpreter will fetch:

  • Table Data: Displays the actual records in the table fetched from the server.
  • Table Structure: Shows the structure or schema of the table (i.e., column names and types).

Once fetched, the table data or structure is presented in a QTableWidget in the right pane, making it easier to visualize and interact with the dataset.

Example Script

Here’s an example of how the Table() command can be used in the custom script:

1
2
Table(Employees, showdata)
Table(Departments, showstructure)

In the above script:

  • The first line fetches and displays the data for the Employees table.
  • The second line fetches and displays the structure of the Departments table.

Both will appear on the right pane of the script editor as dynamic, scrollable tables, providing a clear and interactive view of the information.

3. Function Block Command: Executing Methods Inside the Program

The new Function block feature introduces the ability to call functions (methods) within the program directly from the script. This is a powerful addition, allowing for dynamic interaction between the script and the underlying PyQt6 application. Functions can now be written as part of the program, and by calling them in the script, users can trigger custom logic like displaying data, performing calculations, or updating the UI.

Example Usage:

1
2
3
Function:
     Show_Current_Date(Label4)
End Function

In this example, the Show_Current_Date function is executed when the script is run. It updates Label1 with the current date and time, making it an ideal way to showcase dynamic updates within the user interface.

This function execution capability adds flexibility to the script and enables custom behaviors to be attached to script commands. As a developer, you can define any method and trigger it from within the script block.

4. Upcoming Features: CRUD Operations and Custom Function Creation

In the next phase of the project, we aim to extend this functionality even further. The next enhancement will focus on enabling CRUD (Create, Read, Update, Delete) operations directly from the script. Users will be able to write commands in the script editor that allow them to:

  • Insert new records into the database.
  • Update existing records.
  • Delete records.
  • Perform custom queries.

Additionally, we will introduce a function creation feature, allowing users to define their own functions within the script. This will make the editor even more powerful, giving users control over the logic of their application directly through scripting.

Conclusion

With these new features, the Custom Script Editor is evolving into a more versatile and dynamic tool. By enabling table data and structure display, along with the ability to execute functions, we've significantly enhanced the editor’s interactivity and functionality. In the next phase, we’ll take another big leap by adding database CRUD operations and empowering users to create custom functions, pushing the boundaries of what the script editor can do.

You may download the project on my github page: Custom Script Editor

Stay tuned for the next update!

Saturday, September 21, 2024

Custom Script Editor Project: Adding More Features(3rd Enhancement)

 Introduction

In this blog post, we'll discuss the recent enhancements made to our Script Editor project, which now includes a Flask application for better script management. And as usual, if have not read the previous posts, here are the links: Enhancing the Custom Scripting Language Interpreter in Python with PyQt6 and Building a Custom Scripting Language in PyQt6 for Dynamic Widget Rendering. These enhancements allow users to retrieve scripts from a server, save new scripts to the server, and a new optional parameter, "COMP," for our text input. Let's dive into these exciting features!

Here is the new screen:


New Flask Application

We've introduced a Flask application that serves as the backend for our Script Editor. The primary functions of this application are:

1. Retrieving Scripts: Users can now fetch script files directly from the server using a simple API endpoint.

1
2
3
@app.route('/get-script/<script_name>', methods=['GET'])
def get_script(script_name):
    # Logic to retrieve the script from the server

 2. Saving Scripts: Users can also save their scripts to the server through another endpoint. This functionality is crucial for ensuring that users can maintain their work across sessions.

1
2
3
@app.route('/save-script/<script_name>', methods=['POST'])
def save_script(script_name):
    # Logic to save the script content

These features are implemented in a straightforward manner, making it easy for the frontend to communicate with the server.

Enhanced Text Input with "COMP" Parameter

A significant enhancement to the script editor is the addition of an optional "COMP" parameter for the text input fields. When this parameter is included, a button appears next to the input field. Clicking this button opens a dialog box that displays a list of companies. With this feature, developers can save time and effort when coding the selection of companies. By automating this process, they can focus on other critical aspects of their projects, ultimately enhancing productivity and streamlining workflows. This not only leads to faster development cycles but also minimizes the potential for errors, ensuring a more efficient and reliable application.

Here is the new script format:

1
TextInput Text1(COMP,Company Code)

Here's how the new functionality works:
  •  Text Input Creation: When defining a text input in the script, adding "COMP" will trigger the creation of a button beside it.

1
self._create_text_input(line, horizontal_layout)

  • Dialog for Company Selection: When the button is clicked, it opens a dialog displaying the list of companies, enabling users to select one easily.

1
2
3
4
def open_companies_dialog(self):
    dialog = Companies(initial_settings)
    if dialog.exec() == QDialog.DialogCode.Accepted:
        # Logic to handle selected company

Future Enhancements

Currently, data retrieval for the list of companies is handled within the script editor. However, we plan to enhance this by retrieving the data directly from the server through the Flask application. This improvement will ensure that the script editor app remains dynamic and fully integrated with the backend.

Conclusion

The integration of the Flask application into our Script Editor project marks a significant step toward a more robust and user-friendly tool. With the ability to save and retrieve scripts from a server and the new company selection feature, users can expect an improved experience. Stay tuned for further updates as we continue to enhance this project!

You may download the project on my github page: Custom Script Editor

Friday, September 20, 2024

Enhancing the Custom Scripting Language Interpreter in Python with PyQt6

 In this blog post, we’ll dive into the enhancements made to the custom scripting language and its interpreter built using Python and PyQt6. If you have not read it yet, click here(Building a Custom Scripting Language in PyQt6 for Dynamic Widget Rendering) This project allows users to dynamically generate UI components based on a simple, user-friendly script. The focus is on key improvements such as the command execution, clearing the right pane before displaying new widgets, saving widgets into a registry for better control, and additional functionality like widget alignment and action handling.

Let's walk through the enhancements and improvements in this project:

Key Enhancements

1. Clearing the Right Pane Before Displaying New Widgets

Before the interpreter renders new widgets based on the script, we ensure the right pane is cleared. This prevents residual widgets from previous script executions from cluttering the view.

1
2
3
4
5
6
# Remove all widgets from the layout
while layout.count():
    item = layout.takeAt(0)  # Get the first item from the layout
    widget = item.widget()   # Get the widget from the layout item
    if widget is not None:
        widget.setParent(None)  # Remove the widget from the layout

This block of code ensures that the right pane starts fresh every time a new script is run, improving the user experience and maintaining a clean interface.

2. Action Handling When Buttons Are Clicked

One of the most significant enhancements is adding the ability to define actions within a script. This is accomplished through the use of Command and End Command blocks. The actions are linked to widgets, primarily buttons, allowing for dynamic behavior based on user interaction.

1
2
3
4
# Command for button click
Command:
    Button_Clicked:Submit:print Text1:Label2
End Command

The above block of code defines that when the "Submit" button is clicked, it prints the content of Text1 and updates the text in Label2. This flexibility opens up many possibilities, such as form handling, real-time updates, and interactive elements in the generated UI.

3. Saving All Widgets in a Widget Registry

To maintain control over the dynamically created widgets, we store them in a registry. This allows for easy access, modification, and action application based on widget names provided in the script.

1
self.widget_registry = {}  # Store references to widgets by name

Each widget (e.g., buttons, labels, text inputs) is stored in this registry when created. This makes it easy to retrieve them later when executing commands like updating text or handling button clicks.

For example, here's how we register a button:

1
2
button_name = label.strip()
self.widget_registry[button_name] = button  # Register button by name

4. Support for Widget Alignment

Labels in the script now support text alignment, allowing the user to specify whether the label’s text should be left, center, or right aligned. This provides more control over the layout of the generated UI.

1
label.setAlignment(Qt.AlignmentFlag.AlignCenter)  # Center alignment

The script syntax supports this enhancement:

1
Label Label2(Result:Center)

This creates a label with the text "Result" centered in its container.

5. Dynamic Command Parsing

The custom scripting language includes commands like Button_Clicked, which are dynamically parsed and executed. Here’s how it works:

 1
 2
 3
 4
 5
 6
 7
 8
 9
10
11
def _parse_command(self, line):
    if line.startswith("Button_Clicked"):
        parts = line.split(':')
        button_name = parts[1]
        action = parts[2]
        target = parts[3]
        self.commands.append({
            "button_name": button_name,
            "action": action,
            "target": target
        })

This method parses the button click command and stores it, allowing for delayed execution when the button is actually clicked. The action is bound using partial, capturing both the action and the target widget to apply the changes when the button is clicked.

1
button.clicked.connect(partial(self._execute_action, action, target))

6. Command Execution Based on Script Instructions

When a button is clicked, the associated command is executed. For example, the action to print the content of a text input is defined and applied dynamically:

1
2
3
4
5
6
7
8
9
def _execute_action(self, action, target):
    if action.startswith("print"):
        widget = self.widget_registry.get(target_widget)
        if isinstance(widget, QLineEdit):
            print(widget.text())
            if target:
                widget1 = self.widget_registry.get(target)
                if isinstance(widget1, QLabel):
                    widget1.setText(widget.text())

In this example, the Submit button prints the content of a QLineEdit and updates a QLabel with the same text. This demonstrates the ability to build interactive forms and real-time updates within the custom UI.

Sample Script

Here’s an example script demonstrating all of the new features:

1
2
3
4
5
6
7
8
Label Label1(Enter text)
TextInput Text1(Type text here)
Buttons Submit
Label Label2(Result:Center)

Command:
    Button_Clicked:Submit:print Text1:Label2
End Command

And here is the rendered screen:



This script creates a simple form with a text input and a submit button. Upon clicking the "Submit" button, the text from Text1 is printed and displayed in Label2, which is centered.

Conclusion

By expanding the custom script interpreter, we've added significant flexibility and functionality. Users can now define actions, align labels, and work with a cleaner interface thanks to automatic widget clearing. The widget registry ensures full control over the dynamically created UI elements, making this tool powerful for building custom user interfaces without needing to write traditional PyQt code.

These enhancements pave the way for building more complex applications with minimal effort, offering a higher level of abstraction for UI development. Whether you’re designing forms, dashboards, or simple tools, this script-based approach makes development faster and more intuitive.

And here is the complete program listing:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
import sys
from PyQt6.QtWidgets import (
    QApplication, QMainWindow, QSplitter, QTextEdit, QPushButton, 
    QVBoxLayout, QHBoxLayout, QWidget, QScrollArea, QLabel, QLineEdit, 
    QSpacerItem, QSizePolicy
)
from PyQt6.QtCore import Qt
from functools import partial
import pdb
class ScriptInterpreter:
    
    def __init__(self, script, preview_container):
        self.script = script
        self.preview_container = preview_container
        self.widget_registry = {} # Store references to widgets by name
        self.commands = []  # Store commands to be executed

    def interpret(self):
        # Clear previous widgets
        layout = self.preview_container.layout()
        if layout:
            for i in reversed(range(layout.count())):
                widget_to_remove = layout.itemAt(i).widget()
                if widget_to_remove is not None:
                    widget_to_remove.setParent(None)

        
        end_horizontal_found = True
        in_command_block = False
        
        # Parse the script line by line
        lines = self.script.splitlines()
        for line in lines:
            
            line = line.strip()
           

            
            
            if line.startswith("Buttons") and end_horizontal_found == True:
                self._create_button(line)
            elif line.startswith("Horizontal:"):
                # Start a new horizontal layout
                horizontal_layout = QHBoxLayout()
                horizontal_layout.setSpacing(10)
                self._add_hbox_layout_to_container(horizontal_layout)
                end_horizontal_found = False
            
                
                
            
                
            elif line.startswith("Buttons") and end_horizontal_found == False:
                self._create_button(line)
                
            elif line.startswith("Label") and end_horizontal_found == False:
                self._create_label(line, horizontal_layout)
               
            elif line.startswith("TextInput") and end_horizontal_found == False:
                self._create_text_input(line, horizontal_layout)
                
                
            
                
            elif line.startswith("Label") and end_horizontal_found == True:
                    self._create_label(line)
                    
            elif line.startswith("TextInput") and end_horizontal_found == True:
                    self._create_text_input(line)
                    
            elif line.startswith("End Horizontal"):
                end_horizontal_found = True
            # Check for Command blocks
            elif line.startswith("Command:"):
                #pdb.set_trace() 
                in_command_block = True
                continue
            
            elif line.startswith("End Command"):
                in_command_block = False
                self._apply_commands()
                continue
            elif in_command_block == True :
                self._parse_command(line)
                
                continue
                

        

        # Add spacer to push everything to the top
        spacer = QSpacerItem(20, 40, QSizePolicy.Policy.Minimum, QSizePolicy.Policy.Expanding)
        layout.addItem(spacer)
    def _parse_command(self, line):
        """
        Parse commands from the Command block.
        """
        #pdb.set_trace()
        if line.startswith("Button_Clicked"):
            parts = line.split(':')
            
            button_name = parts[1]
            
            action = parts[2]
            target = parts[3]
            self.commands.append({
                "button_name": button_name,
                "action": action,
                "target": target
            })
            

    def _apply_commands(self):
        #pdb.set_trace()
        """
        Apply the parsed commands to the corresponding widgets.
        """
        for command in self.commands:
            button_name = command["button_name"]
            action = command["action"]
            target = command["target"]
            
            action = action.strip()
            button_name= button_name.strip()
            target = target.strip()
            
        
            # Retrieve the button from the registry
            button = self.widget_registry.get(button_name.strip())
            
        
            if button is not None:
                # Use lambda to capture the action argument properly
                #button.clicked.connect(lambda _, a=action: self._execute_action(a))
                button.clicked.connect(partial(self._execute_action, action, target))

    def _execute_action(self, action, target):
        #pdb.set_trace()
        """
        Execute the action when the button is clicked.
        """
        if action.startswith("print"):
            # Extract the target of the print command
            
            _, target_widget = action.split(maxsplit=1)
            
            widget = self.widget_registry.get(target_widget)
            
            if isinstance(widget, QLineEdit):
                # Print the content of the text input
                print(widget.text())
                if target:
                    widget1 = self.widget_registry.get(target)
                    if isinstance(widget1, QLabel):
                        widget1.setText(widget.text())

    def _create_button(self, line, layout=None):
        _, label = line.split(maxsplit=1)
        button = QPushButton(label)
        button.setFixedHeight(40)  # Set fixed height
        button_name = label.strip()
        self.widget_registry[button_name] = button  # Register button by name
        if layout != None:
            layout.addWidget(button)
        else:
            self.preview_container.layout().addWidget(button)

    def _create_label(self, line, layout=None):
        # Extract label information and alignment
        label_parts = line.split("(")
        label_name = label_parts[0].split()[1].strip()  # Label name (like Label1)
    
        if len(label_parts) > 1:
            text_and_align = label_parts[1].rstrip(")").split(":")
            label_text = text_and_align[0].strip()  # Label text (like "Result")
            alignment = text_and_align[1].strip().lower() if len(text_and_align) > 1 else 'left'
        else:
            label_text = ""
            alignment = 'left'
    
        label = QLabel(label_text)
        label.setFixedHeight(40)  # Set fixed height
        
        # Register the text input by its name
        self.widget_registry[label_name] = label

        # Set alignment based on the provided value (left, center, right)
        if alignment == 'center':
            label.setAlignment(Qt.AlignmentFlag.AlignCenter)
        elif alignment == 'right':
            label.setAlignment(Qt.AlignmentFlag.AlignRight)
        else:
            label.setAlignment(Qt.AlignmentFlag.AlignLeft)
            
        if layout != None:
            layout.addWidget(label)
        else:
            self.preview_container.layout().addWidget(label)

    def _create_text_input(self, line, layout=None):
        
        # Extract the name and placeholder from the line
        name_placeholder = line[len("TextInput "):]  # Remove "TextInput " part
        name, placeholder = name_placeholder.split("(", 1)
        name = name.strip()
        placeholder = placeholder.rstrip(")")  # Remove closing parenthesis
        
        text_input = QLineEdit()
        text_input.setPlaceholderText(placeholder)
        text_input.setFixedHeight(40)  # Set fixed height

        # Register the text input by its name
        self.widget_registry[name] = text_input

        
        if layout != None:
            layout.addWidget(text_input)
        else:
            self.preview_container.layout().addWidget(text_input)

    def _add_hbox_layout_to_container(self, layout):
        # Create a container widget for the HBoxLayout
        hbox_widget = QWidget()
        hbox_widget.setLayout(layout)
        #hbox_widget.setFixedHeight(50)  # Set a fixed height for the HBoxLayout widget
        self.preview_container.layout().addWidget(hbox_widget)


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("Custom Scripting Language Interpreter")
        self.setGeometry(100, 100, 800, 600)

        # Create a QSplitter to split left and right sections
        self.splitter = QSplitter(Qt.Orientation.Horizontal, self)

        # Left Pane: TextEdit for entering script and a button to test it
        self.left_widget = QWidget()
        self.left_layout = QVBoxLayout(self.left_widget)

        self.text_edit = QTextEdit()
        self.text_edit.setPlaceholderText("Enter your script here...")

        self.test_button = QPushButton("Test Script")
        self.test_button.clicked.connect(self.run_script)

        self.left_layout.addWidget(self.text_edit)
        self.left_layout.addWidget(self.test_button)

        # Right Pane: ScrollArea to preview the widgets from the script
        self.scroll_area = QScrollArea()
        self.scroll_area.setWidgetResizable(True)

        self.preview_container = QWidget()
        self.preview_container.setLayout(QVBoxLayout())
        self.scroll_area.setWidget(self.preview_container)

        # Add left and right panes to the splitter
        self.splitter.addWidget(self.left_widget)
        self.splitter.addWidget(self.scroll_area)
        splitter_sizes = [int(0.30 * self.width()), int(0.70 * self.width())]
        self.splitter.setSizes(splitter_sizes)

        # Set splitter as the central widget of the main window
        self.setCentralWidget(self.splitter)
    def get_all_widgets(self, scroll_area):
        
        # Get the container widget inside the QScrollArea
        container = scroll_area.widget()

        # Check if the container has a layout
        if container.layout() is not None:
            # Get all widgets from the container
            widgets = []
            layout = container.layout()
            for i in range(layout.count()):
                widget = layout.itemAt(i).widget()
                if widget is not None:
                    widgets.append(widget)
            return widgets
        else:
            return []

    def run_script(self):
        # Get the layout of the container inside the scroll area
        layout = self.preview_container.layout()
    
        # Remove all widgets from the layout
        while layout.count():
            item = layout.takeAt(0)  # Get the first item from the layout
            widget = item.widget()   # Get the widget from the layout item
            if widget is not None:
                widget.setParent(None)  # Remove the widget from the layout
        
        # Get the script from the text editor
        script = self.text_edit.toPlainText()
        

        # Interpret and render the widgets in the right pane
        interpreter = ScriptInterpreter(script, self.preview_container)
        interpreter.interpret()


# Main application
app = QApplication(sys.argv)
main_window = MainWindow()
main_window.show()

# Run the PyQt6 event loop
sys.exit(app.exec())

Thursday, September 19, 2024

Building a Custom Scripting Language in PyQt6 for Dynamic Widget Rendering

 In this tutorial, we’ll walk through creating a custom scripting language interpreter that renders widgets dynamically in a PyQt6-based application. This approach simulates a simple browser-like software where widgets (such as buttons, labels, and text inputs) are rendered based on commands from a text script.

The rationale behind this project stems from a common challenge faced when distributing software as executable (EXE) files: updating features. Typically, software developers require users or customers to download the latest EXE files to get access to new features or updates. This approach can be inconvenient and time-consuming for users, who are forced to repeatedly download and install new versions of the software.

To address this issue, our solution involves distributing a generic script interpreter instead of a static EXE file. With this setup, the core interpreter remains the same, while the functionality and features can be dynamically updated through script files. This allows for seamless updates, as users only need to download the updated script without modifying the base software. Furthermore, users are given the flexibility to customize the software to suit their specific needs by editing or adding to the script.

This method not only enhances the user experience by simplifying updates, but it also fosters a more adaptable and customizable software environment, empowering users to tailor the solution to their unique requirements.

We will explore how to:

  • Create a PyQt6 application with a text editor and preview pane.
  • Design a simple scripting language syntax.
  • Interpret the script and render the widgets dynamically.
  • This is the expected output:

The Custom Script Syntax

Our custom scripting language will be simple and focus on three types of widgets:

  1. Labels: Display a static text label.
  2. TextInput: Create a text input field.
  3. Button: Add a clickable button.

Additionally, we will support:

  • Horizontal Layouts: Group widgets horizontally for better control over their layout.

Here is a basic example of the script syntax:

1
2
3
4
5
6
7
8
9
Label Enter your name:
TextInput Your name
Button Submit

Horizontal:
    Label Username:
    TextInput Enter your username
    Button Login
End Horizontal

The syntax is designed to be readable:

  • Commands are prefixed with the widget type (e.g., Label, TextInput, or Button).
  • The content for each widget (such as label text or input placeholder) follows the command.
  • Horizontal layout starts with the Horizontal: command and ends with End Horizontal.

PyQt6 Application Overview

The application consists of two main sections:

  1. Left Pane: A QTextEdit where users can input the script, and a button to execute the script.
  2. Right Pane: A QScrollArea to display the rendered widgets based on the script.

Here’s how the application layout looks:

  • The left pane allows you to write and edit your script.
  • The right pane shows the dynamically generated interface as defined by the script.

The Code

Let's dive into the full code:

  1
  2
  3
  4
  5
  6
  7
  8
  9
 10
 11
 12
 13
 14
 15
 16
 17
 18
 19
 20
 21
 22
 23
 24
 25
 26
 27
 28
 29
 30
 31
 32
 33
 34
 35
 36
 37
 38
 39
 40
 41
 42
 43
 44
 45
 46
 47
 48
 49
 50
 51
 52
 53
 54
 55
 56
 57
 58
 59
 60
 61
 62
 63
 64
 65
 66
 67
 68
 69
 70
 71
 72
 73
 74
 75
 76
 77
 78
 79
 80
 81
 82
 83
 84
 85
 86
 87
 88
 89
 90
 91
 92
 93
 94
 95
 96
 97
 98
 99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
import sys
from PyQt6.QtWidgets import (
    QApplication, QMainWindow, QSplitter, QTextEdit, QPushButton, 
    QVBoxLayout, QHBoxLayout, QWidget, QScrollArea, QLabel, QLineEdit, 
    QSpacerItem, QSizePolicy
)
from PyQt6.QtCore import Qt

class ScriptInterpreter:
    def __init__(self, script, preview_container):
        self.script = script
        self.preview_container = preview_container

    def interpret(self):
        # Clear previous widgets
        layout = self.preview_container.layout()
        if layout:
            for i in reversed(range(layout.count())):
                widget_to_remove = layout.itemAt(i).widget()
                if widget_to_remove is not None:
                    widget_to_remove.setParent(None)

        end_horizontal_found = True
        
        # Parse the script line by line
        lines = self.script.splitlines()
        for line in lines:
            line = line.strip()
            
            # Render Button if not inside a horizontal layout
            if line.startswith("Button") and end_horizontal_found == True:
                self._create_button(line)
                
            # Start horizontal layout
            elif line.startswith("Horizontal:"):
                horizontal_layout = QHBoxLayout()
                horizontal_layout.setSpacing(10)
                self._add_hbox_layout_to_container(horizontal_layout)
                end_horizontal_found = False
            
            # Render Button inside horizontal layout
            elif line.startswith("Button") and end_horizontal_found == False:
                self._create_button(line, horizontal_layout)
                
            # Render Label inside horizontal layout
            elif line.startswith("Label") and end_horizontal_found == False:
                self._create_label(line, horizontal_layout)
                
            # Render TextInput inside horizontal layout
            elif line.startswith("TextInput") and end_horizontal_found == False:
                self._create_text_input(line, horizontal_layout)
                
            # Render Label outside horizontal layout
            elif line.startswith("Label") and end_horizontal_found == True:
                self._create_label(line)
                
            # Render TextInput outside horizontal layout
            elif line.startswith("TextInput") and end_horizontal_found == True:
                self._create_text_input(line)
                
            # End horizontal layout
            elif line.startswith("End Horizontal"):
                end_horizontal_found = True

        # Add spacer to push everything to the top
        spacer = QSpacerItem(20, 40, QSizePolicy.Policy.Minimum, QSizePolicy.Policy.Expanding)
        layout.addItem(spacer)

    def _create_button(self, line, layout=None):
        _, label = line.split(maxsplit=1)
        button = QPushButton(label)
        button.setFixedHeight(40)  # Set fixed height
        if layout:
            layout.addWidget(button)
        else:
            self.preview_container.layout().addWidget(button)

    def _create_label(self, line, layout=None):
        _, text = line.split(maxsplit=1)
        label = QLabel(text)
        label.setFixedHeight(40)  # Set fixed height
        if layout:
            layout.addWidget(label)
        else:
            self.preview_container.layout().addWidget(label)

    def _create_text_input(self, line, layout=None):
        _, placeholder = line.split(maxsplit=1)
        text_input = QLineEdit()
        text_input.setPlaceholderText(placeholder)
        text_input.setFixedHeight(40)  # Set fixed height
        if layout:
            layout.addWidget(text_input)
        else:
            self.preview_container.layout().addWidget(text_input)

    def _add_hbox_layout_to_container(self, layout):
        hbox_widget = QWidget()
        hbox_widget.setLayout(layout)
        self.preview_container.layout().addWidget(hbox_widget)


class MainWindow(QMainWindow):
    def __init__(self):
        super().__init__()

        self.setWindowTitle("Custom Scripting Language Interpreter")
        self.setGeometry(100, 100, 800, 600)

        # Create a QSplitter to split left and right sections
        self.splitter = QSplitter(Qt.Orientation.Horizontal, self)

        # Left Pane: TextEdit for entering script and a button to test it
        self.left_widget = QWidget()
        self.left_layout = QVBoxLayout(self.left_widget)

        self.text_edit = QTextEdit()
        self.text_edit.setPlaceholderText("Enter your script here...")

        self.test_button = QPushButton("Test Script")
        self.test_button.clicked.connect(self.run_script)

        self.left_layout.addWidget(self.text_edit)
        self.left_layout.addWidget(self.test_button)

        # Right Pane: ScrollArea to preview the widgets from the script
        self.scroll_area = QScrollArea()
        self.scroll_area.setWidgetResizable(True)

        self.preview_container = QWidget()
        self.preview_container.setLayout(QVBoxLayout())
        self.scroll_area.setWidget(self.preview_container)

        # Add left and right panes to the splitter
        self.splitter.addWidget(self.left_widget)
        self.splitter.addWidget(self.scroll_area)
        splitter_sizes = [int(0.30 * self.width()), int(0.70 * self.width())]
        self.splitter.setSizes(splitter_sizes)

        # Set splitter as the central widget of the main window
        self.setCentralWidget(self.splitter)

    def run_script(self):
        # Get the script from the text editor
        script = self.text_edit.toPlainText()
        print("Script to be interpreted:")
        print(script)

        # Interpret and render the widgets in the right pane
        interpreter = ScriptInterpreter(script, self.preview_container)
        interpreter.interpret()


# Main application
app = QApplication(sys.argv)
main_window = MainWindow()
main_window.show()

# Run the PyQt6 event loop
sys.exit(app.exec())

Script Syntax Breakdown

  1. Button: Adds a button to the window.
    • Example: Button Submit
  2. Label: Adds a label with the specified text.
    • Example: Label Enter your name:
  3. TextInput: Adds a text input field with a placeholder.
    • Example: TextInput Your name
  4. Horizontal: Groups widgets in a horizontal layout, terminated by End Horizontal.
    • Example:

1
2
3
4
5
Horizontal:
    Label Username:
    TextInput Enter your username
    Button Login
End Horizontal

Methodology

The overall methodology used in this program is based on dynamic user interface generation:

  1. Script-Driven UI Creation: Users define the layout and content of the user interface through a custom script, which is then interpreted and translated into PyQt6 widgets. This separates the UI logic from the rendering logic.

  2. Component-Based Approach: Each widget type (Button, Label, TextInput) is encapsulated in separate methods for modularity. This makes the code easy to extend—additional widgets or layouts can be added by simply adding new methods in the ScriptInterpreter class.

  3. Event-Driven Execution: The script is executed when the user presses the “Test Script” button. PyQt6's event loop then dynamically updates the preview pane.

Conclusion

This project demonstrates how to build a custom scripting language interpreter in PyQt6 to render widgets dynamically. By separating the script definition from the rendering logic, this approach allows users to easily define complex layouts with simple text-based commands.

Potential Extensions

  • More Widget Types: You could extend the script to support additional PyQt6 widgets such as dropdowns, checkboxes, or sliders.
  • Advanced Layouts: Implement vertical layouts, grids, or even nested layouts to give users more control over the UI.
  • Error Handling: Add robust error handling and feedback to help users write correct scripts.