Most organizations that mirror CVEs do so because they are adding Temporal and/or Environmental components to the Base CVSS score. It is likely efficient to store scores from a feed in a traditional RDBMS. However, if an organization needs to combine unstructured data such as news articles, email chains, etc to their CVE data, then a database that optimizes unstructured storage may be worth considering.
Adding overall CVE information (e.g. CVSS Base/Temporal/Environmental scoring) together with TI and asset investigations to MongoDB is one solution for managing all the information in a hybrid structured/unstructured format. The result will be easily searchable and indexable. Starting with simple routine to insert a file, I will demonstrate how to achieve this and ultimately build a simple scoring platform.
Let’s start with a simple piece of code to store the credentials on the file system rather than in the code.
def get_params(filename):
'''read values from a file and get parameters
place your mongodb connect information in this file
server,<yourservername or IP>
username,<username>
password,<mongopassword>
authSource,<db you are authenticating against>
'''
myparams = {}
with open(filename) as myfile:
for line in myfile:
key, value = line.partition("=")[::2]
myparams[key.strip()] = value.strip()
return myparams
Now add a bit of pymongo connection code–see the many documents available on pymongo using a Google search:
import json
from pymongo import MongoClient
def connect_mongo(params):
'''Connect to a mongo instance using params from input
params - a dictionary with user, password, server and database
'''
client = MongoClient(params.get("server"),\
username=params.get("user"),\
password=params.get("password"),\
authSource=params.get("authSource"),\
authMechanism='SCRAM-SHA-1')
return client
Finally, add some code to actually insert the file:
def insert_json_file(client, db, collection, file):
mydb = client[db]
mycol = mydb[collection]
with open(file) as jsonf :
file_data = json.load(jsonf)
if isinstance(file_data, list):
mycol.insert_many(file_data)
else:
mycol.insert_one(file_data)
client.close()
Most of the above code is available on various MongoDB-Python tutorials for those that want to understand it in more depth. Calling the code and actually inserting a document is simple:
par = get_params("mongo.env")
cli = connect_mongo(par)
insert_json_file(cli, "nvd", "nvdcve", "nvdcve-1.1-2002.json")
This inserts the 2002 JSON file into Mongo. Note that the db I used requires authentication (many MongoDB-Python demonstrations on the web omit authentication).