-
Notifications
You must be signed in to change notification settings - Fork 215
Using the web API
Malcom has a web API that can be queried using any web client (curl, wget, urllib, or the python requests library).
Malcom generates an API key for every user created in the database. The API key can be found in your account settings. Just point your browser to /account/settings. This API key has to be included as a header in your requests. The curl command for this is:
curl <your Malcom host>:8080/api/query/?value=malcom.io -H "X-Malcom-API-Key: <copy and paste your API key here>"If you have redirections, you might want to pass the The -L flag to curl to follow them
Malcom opens some functions that allow for free database query.
Malcom's database can be queried freely launching GET requests to its web API located at /api/query/. Since values may vary from element type to element type, to get an idea of what parameters you can pass, have a look at the /dataset view.
Common parameters include:
-
value:curl <your Malcom host>:8080/api/query/?value=malcom.io -
type:curl <your Malcom host>:8080/api/query/?type=url -
tags:curl <your Malcom host>:8080/api/query/?tags=zeus
The response is a JSON blob, similar to this:
{
"elements": [
{
"last_analysis": {
"$date": 1429550682281
},
"next_analysis": {
"$date": 1429572282281
},
"tags": [
"search"
],
"date_updated": {
"$date": 1429550682292
},
"link_value": "/nodes/value/malcom.io",
"evil": {},
"value": "malcom.io",
"date_created": {
"$date": 1429550681802
},
"refresh_period": 21600,
"_id": {
"$oid": "553536591d41c8b9dd75aa3c"
},
"type": "hostname",
"link_type": "/nodes/type/hostname"
}
],
"total_results": 1,
"fields": [
[
"value",
"Value"
],
[
"type",
"Type"
],
[
"tags",
"Tags"
],
[
"date_updated",
"Updated"
],
[
"date_created",
"Created"
],
[
"last_analysis",
"Analyzed"
]
],
"chrono_query": "0:00:00.001154",
"chrono_count": "0:00:00.000538",
"per_page": 50,
"page": 0
}-
total_results: The number of results returned by the query. This is currently limited to 500. -
fields: The fields (and matching field label) corresponding to the type of the first element returned by the query -
elements: An array of elements returned by the query. These can be of different types, and usually have different fields. -
chrono_queryandchrono_count: The time it took to run the query and to count results.
If nothing is found, the following is returned:
{
"elements": [],
"total_results": 0,
"fields": [
["value", "Value"],
["type", "Type"],
["tags", "Tags"]
],
"chrono_query": "0:00:00.450309",
"chrono_count": "0:00:00.001948",
"per_page": 50,
"page": 0
}This takes all nodes matching an input query and returns all immediate neighbors for such nodes, and corresponding links.
For example, a GET request to malcom:8080/api/neighbors/?value=malcom.io will yield:
{
"nodes": [
{
"last_analysis": {
"$date": 1405338044878
},
"next_analysis": {
"$date": 1405359644878
},
"tags": [],
"fields": [
[ "value", "Value"],
[ "type", "Type"],
[ "tags", "Tags"],
[ "date_updated", "Updated"],
[ "date_created", "Created"],
[ "last_analysis", "Analyzed"]
],
"value": "bevrifuli.geohats.com",
"date_created": {
"$date": 1405316440420
},
"refresh_period": 21600,
"_id": {
"$oid": "53c36d584374d35c84e57e4e"
},
"type": "hostname",
"whois": "Whois resolution failed"
},
{...},
],
"edges": [
{
"attribs": "host",
"src": {
"$oid": "53c36d5a4374d320aae58118"
},
"dst": {
"$oid": "53c36d584374d35c84e57e4e"
},
"first_seen": {
"$date": 1405316442882
},
"_id": {
"$oid": "53c36d5a4374d35c9ee57e4e"
},
"last_seen": {
"$date": 1405316442882
}
},
{...},
]
}The node array and its elements have the same format as before. edges is an array of elements which contain the source (src) element's _id (src), the destination (dst) element's _id, and dates where this specific link was first seen and last seen. All src and dst values in the edges array match an _id in the nodes array.
If nothing is found, an empty response is returned:
{
"nodes": [],
"edges": []
}This takes all nodes matching an input query (in the same format than the /api/query/ API) and will hop from neighbor to neighbor checking if it finds elements that have been tagged evil. When successful, it will return the whole chain. For example, this might come in handy if a sample is contacting a domain that was previously seen hosting malware.
The /api/evil/ API takes a depth parameter (defaults and max to 2), which is the recursion level that the function will use to hop from node to node until it finds an element tagged 'evil'. If the recursion limit is reached and Malcom didn't find anything, it will return an empty result for this element.
For example, a GET request to malcom:8080/api/evil/?value=bevrifuli.geohats.com will yield the following JSON blob:
{
"nodes": [
{...},
],
"edges": [
{
"attribs": "host",
"src": {
"$oid": "53c36d5a4374d320aae58118"
},
"dst": {
"$oid": "53c36d584374d35c84e57e4e"
},
"first_seen": {
"$date": 1405316442882
},
"_id": {
"$oid": "53c36d5a4374d35c9ee57e4e"
},
"last_seen": {
"$date": 1405316442882
}
},
{...},
]
}The node array and its elements have the same format as before. edges is an array of elements which contain the source (src) element's _id (src), the destination (dst) element's _id, and dates where this specific link was first seen and last seen. All src and dst values in the edges array match an _id in the nodes array.
If nothing of interest is found for the given node, empty arrays are returned:
{
"nodes": [],
"edges": []
}Malcom opens some function that allow for querying sniffer sessions.
This will return a list of sniffer sessions. By default, all public sniffer sessions are returned. You can use the following parameters to tune your request.
-
user: Setting this parameter will have your own (private or not) sniffer sessions included in the response. -
private: Setting this parameter will limit results to only session opened by you (public or not) -
page: Responses are limited to 50 results. Increase this parameter to get the following 50 results.
The response to curl -H "X-Malcom-API-Key: xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx" malcom:8080/api/sniffer/sessionlist/ looks like this:
{
"session_list": [
{
"status": "Running",
"name": "dridex",
"packets": 1405,
"public": true,
"edges": 7,
"date_created": {
"$date": 1429551276776
},
"nodes": 5,
"id": "553538ac1d41c8b9dd75aa3e"
}
]
}You can specify three parameters:
This will yield all data related to a given session ID:
{
"status": false,
"packet_count": 1405,
"name": "dridex",
"node_list": [
"553538b61d41c8b9dd75aa49",
"553538ad1d41c8b9dd75aa41",
"553538cf1d41c8b9dd75aa52",
[more node IDs],
"553538ad1d41c8b9dd75aa42",
"553538ec1d41c8b9dd75aa7f",
"553538d01d41c8b9dd75aa55"
],
"id": "553538ac1d41c8b9dd75aa3e",
"filter": "ip and not host 127.0.0.1 and not host 172.16.254.1 and not host 172.16.254.137 ",
"pcap_filename": "553538ac1d41c8b9dd75aa3e-dridex.pcap",
"pcap": true,
"public": true
}Notable fields:
-
status: The session status (running or not) -
node_list: The list of node IDs. Should you need extra details on these, you can issue a new call to/api/query/or/api/sniffer/data/<string:session_id>/?elements=1(see below) -
filter: BPF used for this session -
pcap_filename: The filename where Malcom stored the raw.pcap -
pcap: Is a.pcapfile available for this session?
This should be used when you want the details of elements contained in a sniffing session. To get all elements contained in a sniffing session, use the parameter ?elements=1. This will return a node_list element containing all nodes in the sniffing session.
Example result:
{
"node_list": [
{
"domain": "-",
"last_analysis": {
"$date": 1429551278113
},
"next_analysis": {
"$date": 1429810478113
},
"tags": [
"dridex",
"sniffer"
],
"country": "",
"ISP": "Private IP Address LAN",
"evil": {},
"bgp": "",
"value": "192.168.221.134",
"date_created": {
"$date": 1429551276817
},
"refresh_period": 259200,
"_id": {
"$oid": "553538ac1d41c8b9dd75aa3f"
},
"type": "ip",
"asn": "",
"name": ""
},
{...}
]
}This will filter elements from /api/sniffer/data/<session_id>/?elements=1 only leaving out elements that have the evil tag set.
api.add_resource(SnifferSessionDelete, '/api/sniffer/delete/<session_id>') api.add_resource(SnifferSessionStart, '/api/sniffer/new/', endpoint='malcom_api.session_start')
This function takes the following POST parameters as a request:
-
pcapfile: The pcap file you want to parse. Not specifying this will result in Malcom sniffing from the wire. -
session_name: Session name (required) -
intercept_tls: Intercept TLS traffic? (boolean, default is False) -
public: Share this session with other users (boolean, default is True) -
start: Start sniffing right away (boolean, default is False) -
filter: BPF filter to use for this session
A successful call will return a session ID.
Set the action parameter to start or stop so start or stop a sniffing session.
This will return the full, raw PCAP associated with a session. Use the id parameter from the sessionlist call.
curl -H "X-Malcom-API-Key: xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx-xxxxxxxxxx" malcom:8080/api/sniffer/53c2cfd31d41c857b73536bb/pcap > test_session.pcap