I am building a travel website with django. When a user is typing in the destination city name (or points of interest, like yellow stone), I want to do ajax auto suggestion. The question is how I could get the suggestion database? Is there any web service? Best if it could also support foreign cities. Thanks a lot.
What you want is called a gazetteer database.
The official USGS gazetteer for the USA is available for download.
Two global geocoded databases include:
Geonames has a free list of cities and POI. It includes the USGS gazetteer and lots of other info. You might have to subset their database however, as it might return too many results for you.
Maxmind also have a free database of cities.
take a look at OpenStreetMap there are a lot of cities, pois both in chinease and english
Related
As MS Support recently told me that using a "GET" is much more efficient in RUs usage than a sql query. I'm wondering if I can (within the azure.cosmos python package or a custom HTTP request to the REST API) get a document by its unique 'id' field (for which I generated a GUIDs) without an SQL Query.
Every example shown are using the link/path of the doc which is built with the '_rid' metadata of the document and not the 'id' field set when creating the doc.
I use a bulk upsert stored procedure I wrote to create my new documents and never retrieve the metadata for each one of them (I have ~ 100 millions docs) so retrieving the _rid would be equivalent to retrieving the doc itself.
The reason that the ReadDocument method is so much more efficient than a SQL query is because it uses _rid instead of a user generated field, even the required id field. This is because the _rid isn't just a unique value, it also encodes information about where that document is physically stored.
To give an example of how this works, let's say you are explaining to someone where a party is this weekend. You could use the name that you use for the house "my friend Ryan's house" or you could use the address "123 ThatOne Street Somewhere, WA 11111". They both are unique identifiers, but for someone trying to get there one is way more efficient than the other.
Telling someone to go to your friend's house is like using your own id. It does map to a specific house, but the person will still need to find out where that physically is to get there. Using the address is like working with the _rid field. Based on that information alone they can get to the party location. Of course, in the real world the person would probably need directions, but the data storage in a database is a lot more organized than most city streets so an address is sufficient to go retrieve the document.
If you want to take advantage of this method you will need to find a way to work with the _rid field.
Here in Brazil, the government made an app that you can type in the name of the item, for example "Refined Sugar 2kg" and the app return to you the location of the nearest market that has the cheapest item.
It extends to all sort of products, and I'm thinking about creating an app using that database. It is public, so it's not a crime, but they do not list all the products on the screen right away with categories and etc. The app only have a search bar, and that's it.
How can I get that database? I thought of an crawler (probably python), but all the crawlers that i see has an webpage as a 'target'. I never programmed mobile, so I don't know if a crawler can do that job.
So, what should I study in order to make an tool to extract that information?
"How can I get that database? " The legal way would be to just ask them for it. Their database likely has passwords/etc which means it is not meant for public use outside of the functionality that the application you mentioned provides. Are there ways to access this data without asking the owner of the information? Possibly. But to attain the data in any other way then asking for it and receiving it, then my friend you are entering a grey area. A crawler won't be much help for what you're trying to do.
I'm looking for a way to parse objects out of naturally written text. I have a database of thousands of locations (City, State). As my users write posts, I would like to intelligently find and enrich locations being written about. For example, given the post:
I had a really nice trip to Portland this weekend. It was beautiful and the climbing gyms are second to none.
I'd like to suggest Portland, OR and Portland, ME and ask the user to choose one.
Is there a name for this kind of search? I'm not even sure where to start.
EDIT: I'm currently using Python/Django and MySQL, but suggestions on any technology/platform would be useful.
You will need to use NLP to extract the city (location) from your sentence.
See:
http://www.nltk.org/howto/relextract.html
then run a query against your database:
maybe like:
select city, state from locations_table where city="Portland"
which will give you city, state pairs.
I would like to know how to load all the data in cities_light of Django, I have 42 record when I run python manage.py cities_light --force-import-all or python manage.py cities_light
in settings
CITIES_LIGHT_TRANSLATION_LANGUAGES = ['es']
CITIES_LIGHT_INCLUDE_COUNTRIES = ['CR']
This return 42 records in /cities but I need more than 100
I would like to know how charge more records in the database becouse I have so much code to change if I change cities_light for another library.
I thought that I could make a script for charge the rest of the records for instance a csv, but I need the obligatory fields in the database.
Thanks so much for your help.
Cities Light uses the open Geonames database. So it only knows the cities that are in the Geonames database. You can help and add missing cities there or fix the data on existing entries.
Also, there are different data files for cities from Geonames.
cities1000.zip has all cities with a population >1000
cities5000.zip all cities with a population >5000
cities15000.zip all cities with a population >15000
So maybe you are importing the smallest file that includes only larger towns and cities.
Try to use the http://download.geonames.org/export/dump/[Country_Code].zip
It's the only way I found to load more cities.
You may need to update the CITIES_LIGHT_INCLUDE_CITY_TYPES variable see
http://www.geonames.org/export/codes.html.
I Hope that this will help someone because it took me sometime to figure all this out
I had read about an app called citycounds.fm, which is no longer active, where they made city-based playlists. Unfortunately, I can't seem to find any way to search for tracks by city in the soundcloud api documentation.
Any one know if this is possible?
You can't filter tracks by city. The city is actually stored with the user. So you would have to search for the tracks you want, then perform an additional step to check if the user for each of the tracks is from the city you want.
I wanted to do something similar, but too many users do not have their city saved in their profile so the results are very limited.