smint_api.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Search by Sound SMINT API Tutorial\n",
    "    \n",
    "### by Thomas Lidy \n",
    "\n",
    "This tutorial shows how to use the Search by Sound system using SMINT REST API.\n",
    "\n",
    "The Search by Sound system is a Web service for uploading files for server-side audio analysis, and query the system for similar sounding files.\n",
    "\n",
    "### The API endpoint is: http://musicbricks.ifs.tuwien.ac.at/smintapi/\n",
    "\n",
    "### The Info page is here: http://ifs.tuwien.ac.at/mir/musicbricks/\n",
    "\n",
    "The complete API reference is here: http://ifs.tuwien.ac.at/mir/musicbricks/SMINT/SMINT-API-Documentation_v1.1.1c-SbS-rel1.3.pdf"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pre-Requisites:\n",
    "\n",
    "urllib and httplib2 installed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# in Terminal\n",
    "sudo pip install urllib httplib2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Python Code:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "import urllib, httplib2\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Initialize HTTP connection\n",
    "h = httplib2.Http(\".cache\")\n",
    "\n",
    "# not needed\n",
    "#h.add_credentials(\"user\", \"******\", \"https://api.github.com\")\n",
    "#data = urllib.urlencode({\"name\":\"test\"})\n",
    "#resp, content = h.request(github_url, \"POST\", data)\n",
    "\n",
    "# we need to set the headers to accept xml\n",
    "headers={\"Accept\": \"application/xml\"}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API Call: version\n",
    "This is basically to test if the API endpoint can be reached and returns correct xml."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><version major=\"1\" minor=\"3\" revision=\"999\"/></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "\n",
    "url = \"http://musicbricks.ifs.tuwien.ac.at/smintapi/version\"\n",
    "resp, content = h.request(url, \"GET\", headers=headers)\n",
    "\n",
    "print resp.status\n",
    "print content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API Call: track\n",
    "This is to retrieve all similar tracks to a track given by the internal SMINT ID:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><query id=\"5d0da180b9ff948a31da49f76680b85d\" collection=\"_d\"><result id=\"1001cf76cac651793a3df06188172f6d\"/><result id=\"6ae495f05f58ebba11d34c107d07bf5d\"/><result id=\"868816b234d02d363d748312d2980537\"/><result id=\"c8673a26ac9f7201999570b03989b1f0\"/><result id=\"2a6392be78e6d3934e6c8fbb6a39665d\"/><result id=\"dc6e892b25e8908defa9603266b22bd4\"/><result id=\"92ac25b604036177394a45d7c37a7916\"/><result id=\"947d00fe937e6de9eb39b13606d34fab\"/></query></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "query_smint_id = '5d0da180b9ff948a31da49f76680b85d'\n",
    "\n",
    "base_url = \"http://musicbricks.ifs.tuwien.ac.at/smintapi/track/\" + query_smint_id\n",
    "\n",
    "resp, content = h.request(base_url, \"GET\", headers=headers)\n",
    "\n",
    "print resp.status\n",
    "print content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Options\n",
    "#### Control the number of similar tracks returned:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><query id=\"5d0da180b9ff948a31da49f76680b85d\" collection=\"_d\"><result id=\"6ae495f05f58ebba11d34c107d07bf5d\"/><result id=\"1001cf76cac651793a3df06188172f6d\"/><result id=\"868816b234d02d363d748312d2980537\"/></query></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "url = base_url + \"?count=3\"\n",
    "\n",
    "resp, content = h.request(url, \"GET\", headers=headers)\n",
    "print resp.status\n",
    "print content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Include distance values in the result:\n",
    "Note that distance values have no absolute meaning, they are to be interpreted relatively."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "collapsed": false,
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><query id=\"5d0da180b9ff948a31da49f76680b85d\" collection=\"_d\"><result id=\"1001cf76cac651793a3df06188172f6d\" value=\"105.257261\"/><result id=\"6ae495f05f58ebba11d34c107d07bf5d\" value=\"105.257261\"/><result id=\"868816b234d02d363d748312d2980537\" value=\"105.257261\"/><result id=\"c8673a26ac9f7201999570b03989b1f0\" value=\"105.257261\"/><result id=\"2a6392be78e6d3934e6c8fbb6a39665d\" value=\"105.257261\"/><result id=\"dc6e892b25e8908defa9603266b22bd4\" value=\"120.673364\"/><result id=\"92ac25b604036177394a45d7c37a7916\" value=\"679.496076\"/><result id=\"947d00fe937e6de9eb39b13606d34fab\" value=\"773.475249\"/></query></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "url = base_url + \"?distance_values=true\"\n",
    "\n",
    "resp, content = h.request(url, \"GET\", headers=headers)\n",
    "print resp.status\n",
    "print content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Include external keys in the result:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><query id=\"5d0da180b9ff948a31da49f76680b85d\" external_key=\"/home/lidy/smafe/test/resources/testsmall10percent faster.mp3\" collection=\"_d\"><result id=\"1001cf76cac651793a3df06188172f6d\" external_key=\"/home/lidy/smafe/test/resources/testWrap/testsmall.mp3\"/><result id=\"6ae495f05f58ebba11d34c107d07bf5d\" external_key=\"/home/lidy/smafe/test/resources/testWrap/limit_too_many1.mp3\"/><result id=\"868816b234d02d363d748312d2980537\" external_key=\"/home/lidy/smafe/test/resources/testWrap/limit_too_many2.mp3\"/><result id=\"c8673a26ac9f7201999570b03989b1f0\" external_key=\"/home/lidy/smafe/test/resources/testthird.mp3\"/><result id=\"2a6392be78e6d3934e6c8fbb6a39665d\" external_key=\"/home/lidy/smafe/test/resources/testsmall.mp3\"/><result id=\"dc6e892b25e8908defa9603266b22bd4\" external_key=\"/home/lidy/smafe/test/resources/testWrap/testlong.mp3\"/><result id=\"92ac25b604036177394a45d7c37a7916\" external_key=\"/home/lidy/music/Lamb_-_Wise_Enough.mp3\"/><result id=\"947d00fe937e6de9eb39b13606d34fab\" external_key=\"/home/lidy/music/Lamb_-_We_Fall_in_Love.mp3\"/></query></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "url = base_url + \"?external_keys=true\"\n",
    "\n",
    "resp, content = h.request(url, \"GET\", headers=headers)\n",
    "print resp.status\n",
    "print content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API Call: track_external_key\n",
    "#### Query using external ID\n",
    "\n",
    "This is retrieving similar tracks to a track given by external ID.\n",
    "The external ID is passed when adding a track. It is usually a file or Web URI."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "# Currently not working with URLs - have to find out how to escape the / properly\n",
    "#external_key=\"/home/lidy/music/Lamb_-_Wise_Enough.mp3\"\n",
    "\n",
    "#external_key = urllib.quote_plus(external_key)\n",
    "#print external_key\n",
    "\n",
    "external_key = 'fight_your_fight'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "http://musicbricks.ifs.tuwien.ac.at/smintapi/track_external_key/fight_your_fight\n",
      "200\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><query id=\"\" external_key=\"fight_your_fight\" collection=\"_d\"><status status_code=\"1\">waiting for processing.</status></query></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "base_url = \"http://musicbricks.ifs.tuwien.ac.at/smintapi/track_external_key/\" + external_key\n",
    "print base_url\n",
    "\n",
    "resp, content = h.request(base_url, \"GET\", headers=headers)\n",
    "\n",
    "print resp.status\n",
    "print content"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Same Options as for API Call 'track' are applicable."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API Call: track/add : Add a Track to the Database for Analysis:\n",
    "\n",
    "By this you provide a URL to an MP3 song you wish to be analyzed, it will be added to the database and analyzed subsequently. When the analysis is ready (usually 2 seconds later) it will already be included in any query results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "200\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><track_added smint_track_id=\"774633055580a0aec513c3.69246365\" url=\"http://www.soundpark.at/mp3/2005-07/moped2067_fight_your_fight_137691.mp3\" file=\"/tmp/smafe_dtpJEZ_moped2067_fight_your_fight_137691.mp3\" external_key=\"fight_your_fight\"/></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# add an MP3 from the Web for analysis \n",
    "\n",
    "mp3_url = 'http://www.soundpark.at/mp3/2005-07/moped2067_fight_your_fight_137691.mp3'\n",
    "\n",
    "# the external key is anything you want (to store in your own database), usually a unique ID or the filename\n",
    "external_key = 'fight_your_fight'\n",
    "\n",
    "data = {'external_key': external_key, 'url': mp3_url}\n",
    "body = urllib.urlencode(data)\n",
    "\n",
    "api_url = \"http://musicbricks.ifs.tuwien.ac.at/smintapi/track/add\"\n",
    "resp, content = h.request(api_url, \"PUT\", headers=headers, body=body) # Note: does not work with POST\n",
    "\n",
    "print resp.status\n",
    "print content\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### API Call: track/delete\n",
    "This deletes a track from the online database. It will create a delete job. \n",
    "Once executed, the track can no longer be queried and will no longer show up in any query results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "403\n",
      "<?xml version=\"1.0\" encoding=\"UTF-8\"?>\n",
      "<smint><error code=\"403\">Forbidden</error><errorDetail>235027ERROR:  null value in column \"file_id\" violates not-null constraint</errorDetail></smint>\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# we use e.g. the smint_id of the track added just above\n",
    "smint_id = '774633055580a0aec513c3.69246365'\n",
    "\n",
    "api_url = \"http://musicbricks.ifs.tuwien.ac.at/smintapi/track/delete/\" + smint_id\n",
    "\n",
    "# use HTTP DELETE\n",
    "resp, content = h.request(api_url, \"DELETE\", headers=headers) # Note: does not work with POST\n",
    "\n",
    "print resp.status\n",
    "print content\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### TODO"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# to test: can we add local mp3 files (upload(ed) to server)\n",
    "#mp3= open('/home/lidy/Musik/Lamb_-_Wise_Enough.mp3', 'rb')"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}