Skip to content

Commit

Permalink
feat: send new connections to google sheets spreadsheet
Browse files Browse the repository at this point in the history
  • Loading branch information
@FlorentLvr
FlorentLvr committed Oct 31, 2023
1 parent 18f04c6 commit e44c43f
Show file tree
Hide file tree
Showing 2 changed files with 55 additions and 82 deletions.
4 changes: 2 additions & 2 deletions LinkedIn/LinkedIn_Get_connections_from_network.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@
"tags": []
},
"source": [
"### Setup Variables\n",
"### Setup variables\n",
"**Mandatory**\n",
"\n",
"[Learn how to get your cookies on LinkedIn](https://www.notion.so/LinkedIn-driver-Get-your-cookies-d20a8e7e508e42af8a5b52e33f3dba75)\n",
Expand Down Expand Up @@ -359,4 +359,4 @@
},
"nbformat": 4,
"nbformat_minor": 5
}
}
133 changes: 53 additions & 80 deletions LinkedIn/LinkedIn_Send_connections_to_Google_Sheets_spreadsheet.ipynb
View file
Original file line number Diff line number Diff line change
Expand Up @@ -26,7 +26,7 @@
"tags": []
},
"source": [
"# LinkedIn - Send connections from network to gsheet\n",
"# LinkedIn - Send connections to a Google Sheets spreadsheet\n",
"<a href=\"https://app.naas.ai/user-redirect/naas/downloader?url=https://raw.githubusercontent.com/jupyter-naas/awesome-notebooks/master/LinkedIn/LinkedIn_Send_connections_from_network_to_gsheet.ipynb\" target=\"_parent\"><img src=\"https://naasai-public.s3.eu-west-3.amazonaws.com/Open_in_Naas_Lab.svg\"/></a><br><br><a href=\"https://bit.ly/3JyWIk6\">Give Feedback</a> | <a href=\"https://github.com/jupyter-naas/awesome-notebooks/issues/new?assignees=&labels=bug&template=bug_report.md&title=LinkedIn+-+Send+connections+from+network+to+gsheet:+Error+short+description\">Bug report</a>"
]
},
Expand Down Expand Up @@ -133,87 +133,44 @@
},
{
"cell_type": "markdown",
"id": "wrapped-benjamin",
"id": "28d46265-4963-4fd0-b631-2c4e9c096f24",
"metadata": {
"papermill": {},
"tags": []
},
"source": [
"### Setup LinkedIn\n",
"👉 <a href='https://www.notion.so/LinkedIn-driver-Get-your-cookies-d20a8e7e508e42af8a5b52e33f3dba75'>How to get your cookies ?</a>"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "yellow-footage",
"metadata": {
"papermill": {},
"tags": []
},
"outputs": [],
"source": [
"# Lindekin cookies\n",
"LI_AT = \"AQEDARCNSioDe6wmAAABfqF-HR4AAAF-xYqhHlYAtSu7EZZEpFer0UZF-GLuz2DNSz4asOOyCRxPGFjenv37irMObYYgxxxxxxx\"\n",
"JSESSIONID = \"ajax:12XXXXXXXXXXXXXXXXX\""
]
},
{
"cell_type": "markdown",
"id": "66b77860-a95b-486c-a153-9cab521de365",
"metadata": {
"papermill": {},
"tags": []
},
"source": [
"### Setup your Google Sheet\n",
"👉 Get your spreadsheet URL<br>\n",
"👉 Share your gsheet with our service account to connect : naas-share@naas-gsheets.iam.gserviceaccount.com<br>\n",
"👉 Create your sheet before sending data into it"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "63ba2bd2-cc65-4ddf-8d01-42460006a159",
"metadata": {
"papermill": {},
"tags": []
},
"outputs": [],
"source": [
"# Spreadsheet URL\n",
"SPREADSHEET_URL = \"https://docs.google.com/spreadsheets/d/XXXXXXXXXXXXXXXXXXXX\"\n",
"### Setup variables\n",
"**Mandatory**\n",
"\n",
"# Sheet name\n",
"SHEET_NAME = \"LK_CONNECTIONS\""
]
},
{
"cell_type": "markdown",
"id": "527141d6-d96f-4d45-8a4b-6c6c11d522c9",
"metadata": {
"papermill": {},
"tags": []
},
"source": [
"### Setup Naas"
"[Learn how to get your cookies on LinkedIn](https://www.notion.so/LinkedIn-driver-Get-your-cookies-d20a8e7e508e42af8a5b52e33f3dba75)\n",
"- `li_at`: Cookie used to authenticate Members and API clients\n",
"- `JSESSIONID`: Cookie used for Cross Site Request Forgery (CSRF) protection and URL signature validation\n",
"- `spreadsheet_url`: Google Sheet Spreadsheet URL. Please make sure you shared it with our service account to connect : naas-share@naas-gsheets.iam.gserviceaccount.com\n",
"- `sheet_name`: Sheet name. Please create it before sending data into it\n",
"\n",
"**Optional**\n",
"\n",
"- `limit`: This refers to the quantity of connections you wish to retrieve. If you intend to fetch all connections, use -1. Please note that retrieving all connections might be time-consuming.\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4bfb95c8-1b19-4f5d-8d55-a2c6a68d07cb",
"id": "9719c9a8-fe22-4241-a5b0-3ae1b44f572e",
"metadata": {
"papermill": {},
"tags": []
},
"outputs": [],
"source": [
"naas.scheduler.add(cron=\"0 8 * * *\")\n",
"# Mandatory\n",
"li_at = naas.secret.get(\"LINKEDIN_LI_AT\") or \"YOUR_LINKEDIN_LI_AT\" #example: AQFAzQN_PLPR4wAAAXc-FCKmgiMit5FLdY1af3-2\n",
"JSESSIONID = naas.secret.get(\"LINKEDIN_JSESSIONID\") or \"YOUR_LINKEDIN_JSESSIONID\" #example: ajax:8379907400220387585\n",
"spreadsheet_url = \"https://docs.google.com/spreadsheets/d/1wMEQdG57eKhpSIYdtK9XVmTMbYD6ovIAy2V9XQanom8/edit#gid=579782200\"\n",
"sheet_name = \"New Sheet\"\n",
"\n",
"# -> To delete your scheduler, please uncomment the line below and execute this cell\n",
"# naas.scheduler.delete()"
"# Optional\n",
"limit = 100"
]
},
{
Expand All @@ -235,7 +192,7 @@
"tags": []
},
"source": [
"### Get connections from Google Sheet"
"### Get data from Google Sheet spreadsheet"
]
},
{
Expand All @@ -248,46 +205,49 @@
},
"outputs": [],
"source": [
"df_gsheet = gsheet.connect(SPREADSHEET_URL).get(sheet_name=SHEET_NAME)\n",
"df_gsheet = gsheet.connect(spreadsheet_url).get(sheet_name=sheet_name)\n",
"df_gsheet"
]
},
{
"cell_type": "markdown",
"id": "c74a1ee9-f844-4466-ae72-57f3e317d21d",
"metadata": {
"papermill": {},
"tags": []
},
"id": "c610dc29-c831-4219-abb7-7d9de2737749",
"metadata": {},
"source": [
"### Get new connections"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "5e4cf0e1-6bc0-4bff-b358-4d3bd693bd14",
"id": "a781400f-412b-428e-b9c8-c869f8f6a3a2",
"metadata": {
"papermill": {},
"tags": []
},
"outputs": [],
"source": [
"def get_new_connections(df_gsheet, key=\"PROFILE_URN\"):\n",
"def get_new_connections(\n",
" li_at,\n",
" JSESSIONID,\n",
" limit,\n",
" df_gsheet,\n",
" key=\"PROFILE_ID\"\n",
"):\n",
" profiles = []\n",
" if len(df_gsheet) > 0:\n",
" profiles = df_gsheet[key].unique()\n",
" else:\n",
" df = linkedin.connect(LI_AT, JSESSIONID).network.get_connections(limit=-1)\n",
" df = linkedin.connect(li_at, JSESSIONID).network.get_connections(limit=limit)\n",
" return df\n",
"\n",
" # Get new\n",
" df_new = pd.DataFrame()\n",
" update = True\n",
" while update:\n",
" start = 0\n",
" df = linkedin.connect(LI_AT, JSESSIONID).network.get_connections(\n",
" start=start, count=100, limit=100\n",
" df = linkedin.connect(li_at, JSESSIONID).network.get_connections(\n",
" start=start, count=100, limit=limit\n",
" )\n",
" new_profiles = df[key].unique()\n",
" for i, p in enumerate(new_profiles):\n",
Expand All @@ -299,9 +259,14 @@
" df_new = pd.concat([df_new, df])\n",
" return df_new\n",
"\n",
"\n",
"df_new = get_new_connections(df_gsheet, key=\"PROFILE_URN\")\n",
"df_new"
"df_new = get_new_connections(\n",
" li_at,\n",
" JSESSIONID,\n",
" limit,\n",
" df_gsheet,\n",
")\n",
"print(\"New connections:\", len(df_new))\n",
"df_new.head(1)"
]
},
{
Expand Down Expand Up @@ -336,8 +301,16 @@
},
"outputs": [],
"source": [
"gsheet.connect(SPREADSHEET_URL).send(df_new, sheet_name=SHEET_NAME, append=True)"
"gsheet.connect(spreadsheet_url).send(df_new, sheet_name=sheet_name, append=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "8ca4243a-152b-40c5-b6ea-3bb3f268c1c9",
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
Expand Down

0 comments on commit e44c43f

Please sign in to comment.