Auto-complete search terms

c go java js lua py vbs rkt scm pro

The Google API (and others) can be used to get suggestions to auto-complete a typed query (in progress).

It is intended for JavaScript usage from a web page but can be automated using Python.

Try the following URL in your browser.

http://suggestqueries.google.com/complete/search?client=firefox&q=corovirus

A browser such as Chrome or Firefox will allow the text to be saved as text - but it is a JSON list.

Here is the returned JSON text, formatted to be easier to read.

["corovirus", [ "coronavirus", "coronavirus map", "coronavirus us", "coronavirus in usa", "coronavirus update", "coronavirus tips", "coronavirus vaccine", "coronavirus china", "coronavirus symptoms", "coronavirus italy" ] ]

This manual operation can be automated using, say, Python.

The code shows how a date and time stamp might be created in order to save the results on a periodic basis to a file in a given folder.

Here is the Python code.

import requests
import json
import datetime
import time

dashDateFormat1 = "%Y-%m-%d"
dashTimeFormat1 = "%H-%M"
now1 = datetime.datetime.now()
dashDate1 = now1.strftime(dashDateFormat1)
dashTime1 = now1.strftime(dashTimeFormat1)
timeStamp1 = "{0:s}-{1:s}".format(dashDate1,dashTime1)

termList1 = [
	"coranovirus",
	"covid-19",
	"pandemic",
	]

headersDict1 = {
	"User-agent" : "Mozilla/5.0",
	}
urlBase1 = "http://suggestqueries.google.com/complete/search?client=firefox&q="

print("Suggestions: {0:s}".format(timeStamp1))

for pos1,term1 in enumerate(termList1):
	url1 = "{0:s}{1:s}".format(urlBase1,term1)
	print("")
	print("{0:d}. {1:s} (term)".format(pos1+1,term1))
	print("")
	response1 = requests.get(url1, headers=headersDict1)
	resultList1 = json.loads(response1.content.decode("utf-8"))
	suggestList2 = resultList1[1]

	for pos2,suggest2 in enumerate(suggestList2):
		print("\t{0:d}. \"{1:s}\"".format(pos2+1,suggest2))

Here is the output of the Python code.

Suggestions: 2020-04-01-13-32

1. coranovirus (term)

	1. "coronavirus"
	2. "coronavirus map"
	3. "coronavirus us"
	4. "coronavirus in usa"
	5. "coronavirus update"
	6. "coronavirus tips"
	7. "coronavirus vaccine"
	8. "coronavirus china"
	9. "coronavirus symptoms"
	10. "coronavirus italy"

2. covid-19 (term)

	1. "covid-19 symptoms"
	2. "covid-19 updates usa"
	3. "covid-19 pennsylvania"
	4. "covid-19 map"
	5. "covid-19 updates"
	6. "covid-19 symptom progression"
	7. "covid-19 testing locations"
	8. "covid-19 map us"
	9. "covid-19 vaccine"
	10. "covid-19 diarrhea"

3. pandemic (term)

	1. "pandemic"
	2. "pandemic game"
	3. "pandemic board game"
	4. "pandemics in the 20th century"
	5. "pandemic 2"
	6. "pandemic meaning"
	7. "pandemic game online"
	8. "pandemic netflix trailer"
	9. "pandemic vs epidemic"
	10. "pandemic preparedness plan"

In the result, the first element in the list is the search term provided. The second item is a list of suggestions which, of course, will always start with the search term provided since the suggestions are intended to help complete what is typed for a search (of the Google site).

One way to save results as JSON (or other format) is as follows.

Create a date and time stamp such as 2020-04-01-12 (assuming once per hour, suffix 00 to 23) of the general form yyyy-mm-dd-hh to be used as a file name.
If all the files are in one folder, there is no need for a file name prefix to distinguish files.

A few years ago, I set up a Raspberry Pi to periodically get the suggested search term completion suggestions for a set of search terms. The idea was to wake up every few hours (say 1 to 24 times a day, using a cron job), and check the suggestions.

The analysis then was to determine, over time, what search terms and orderings had changed.