I'm really out of my league here, but I hear that it can be done.
I have a PBX phone server with raw phone data in thousands of different CSV files that are captured every so often (between 8 minutes and 2 hours) that are stored in dated folders.
The only way I can connect to the server is through WinSCP, which just gives me the file structure (looks like Filezilla FTP).
So 2 things:
How would someone go about importing thousands of files into a SQL Server database (2008)?
How would someone go about setting up a timed event to import new CSV files as they are created?
I just need some direction. I don't even know where to start.
Thanks for your help.
If you have the permissions, you can do a BULK INSERT using sqlcmd with a batch file. In addition, you can create an SSIS package.
More information here and here.
1- There are many ways to load csv files into sql server i.e BCP ,BulkLoad, OpenRowSet( Bulk provider) ,SSIS ,Your Custom .Net application
Most popular is Bulk insert. See this blog for tutorial
http://blog.sqlauthority.com/2008/02/06/sql-server-import-csv-file-into-sql-server-using-bulk-insert-load-comma-delimited-file-into-sql-server/
2- Timed event ? again many options but most straight forward -> Setup Nighly SQL jobs
using System;
using System.Data;
using Microsoft.VisualBasic.FileIO;
namespace ReadDataFromCSVFile
{
static class Program
{
static void Main()
{
string csv_file_path=#"C:\Users\Administrator\Desktop\test.csv";
DataTable csvData = GetDataTabletFromCSVFile(csv_file_path);
Console.WriteLine("Rows count:" + csvData.Rows.Count);
Console.ReadLine();
}
private static DataTable GetDataTabletFromCSVFile(string csv_file_path)
{
DataTable csvData = new DataTable();
try
{
using(TextFieldParser csvReader = new TextFieldParser(csv_file_path))
{
csvReader.SetDelimiters(new string[] { "," });
csvReader.HasFieldsEnclosedInQuotes = true;
string[] colFields = csvReader.ReadFields();
foreach (string column in colFields)
{
DataColumn datecolumn = new DataColumn(column);
datecolumn.AllowDBNull = true;
csvData.Columns.Add(datecolumn);
}
while (!csvReader.EndOfData)
{
string[] fieldData = csvReader.ReadFields();
//Making empty value as null
for (int i = 0; i < fieldData.Length; i++)
{
if (fieldData[i] == "")
{
fieldData[i] = null;
}
}
csvData.Rows.Add(fieldData);
}
}
}
catch (Exception ex)
{
}
return csvData;
}
}
}
Related
My team want to implement a ASP.NET CORE Web API based micro service with a plan to replace bulk copy program utility. Currently we are using BCP utility to return 200,000 rows with 30 columns. The data is returned in csv format.
We created a restful endpoint and using ADO.NET we are connecting to SQL server to extract same volume of data. Here is the code:
using (SqlConnection myConnection = new SqlConnection(con))
{
string oString = "Select * from Employees where runid = 1";
SqlCommand oCmd = new SqlCommand(oString, myConnection);
myConnection.Open();
using (SqlDataReader oReader = oCmd.ExecuteReader())
{
while (oReader.Read())
{
//Read data here
}
}
}
With this code, I am getting memory exceptions..
What is the best way to fix this issue considering in future I will get request to return higher volume of data with more users making simultaneous request. I am open to implementing this solution using C#, Java, Python or NodeJs.
The following code streams the data directly from the database, so it should be quite memory efficient and performant. It is using the Sylvan Csv library functionality to create csv records directly from the SqlDataReader.
using Microsoft.AspNetCore.Mvc;
using Microsoft.AspNetCore.Mvc.Infrastructure;
using Microsoft.Data.SqlClient;
using Sylvan.Data.Csv;
...
[HttpGet(Name = "Get")]
public IActionResult Get()
{
return new FileCallbackResult("text/csv", async (outputStream, _) =>
{
using (var myConnection = new SqlConnection(_configuration["ConnectionStrings"]))
{
var cmdText = "Select * from Employees where runid = 1";
var command = new SqlCommand(cmdText, myConnection);
myConnection.Open();
using (SqlDataReader oReader = await command.ExecuteReaderAsync())
{
var streamWriter = new StreamWriter(outputStream);
var csvDataWriter = CsvDataWriter.Create(streamWriter);
await csvDataWriter.WriteAsync(oReader);
}
}
})
{
FileDownloadName = "employees.csv"
};
}
FileCallbackResult is from: https://github.com/StephenClearyExamples/AsyncDynamicZip/blob/net6-ziparchive/Example/src/WebApplication/FileCallbackResult.cs
You can read about it here: https://blog.stephencleary.com/2016/11/streaming-zip-on-aspnet-core.html
I absolutely love using Node.JS for my web projects but i also love using Python at the same time, so i have been wondering, if is it possible to run Python scripts from Node, for example my Node.JS backend calls a python script to retrive some data from a SQL Database and gives it back to Node (i know i can do that all in Node but just using an example). I thought about doing it this way
1.Node creates a Json files which contains the variable the Python script will use(for example name:Jon birthDate:1996)
2.It runs a python script that reads those variables from that Json file(so it searches for Jon born in 1996)
3.It deletes the Json file when its done
4.Rinse and Repeat
Would this be a good and safe way of doing this type of thing or are there any other ways of running and "modifiying" the Python script?
I had to build a node app, but for the backend calculations, it seemed better to use NumPy and Pandas, so I did what you are doing... in the end I found an npm package called python-shell.
It works via promises, so I'd call my calculation script, and also would do some clean up with another Python script if there were any errors.
Worked pretty good. Code was something like this:
const PythonShell = require('python-shell');
router.get('/calc', (req, res) => {
PythonShell.run('py/calculate.py', {pyPath: pyPath , args: [tmpPath],}, function(err, results) {
if (err) {
PythonShell.run('py/clean_up.py', {pyPath: pyPath, args: [tmpPath2]}, function(err, results2) {
if (err) throw err;
res.json(
{
message: "error: Running clean up",
ang: 0,
vec: 0,
}
);
});
} else {
let data = JSON.parse(results);
let message = data[0];
let ang = data[1];
let vec = data[2];
res.json(
{
message: message,
ang: ang,
vec: vec,
}
);
}
});
});
I am trying to get my friend name and ids with Graph API v2.0, but data returns empty:
{
"data": [
]
}
When I was using v1.0, everything was OK with the following request:
FBRequest* friendsRequest = [FBRequest requestForMyFriends];
[friendsRequest startWithCompletionHandler: ^(FBRequestConnection *connection,
NSDictionary* result,
NSError *error) {
NSArray* friends = [result objectForKey:#"data"];
NSLog(#"Found: %i friends", friends.count);
for (NSDictionary<FBGraphUser>* friend in friends) {
NSLog(#"I have a friend named %# with id %#", friend.name, friend.id);
}
}];
But now I cannot get friends!
In v2.0 of the Graph API, calling /me/friends returns the person's friends who also use the app.
In addition, in v2.0, you must request the user_friends permission from each user. user_friends is no longer included by default in every login. Each user must grant the user_friends permission in order to appear in the response to /me/friends. See the Facebook upgrade guide for more detailed information, or review the summary below.
If you want to access a list of non-app-using friends, there are two options:
If you want to let your people tag their friends in stories that they publish to Facebook using your App, you can use the /me/taggable_friends API. Use of this endpoint requires review by Facebook and should only be used for the case where you're rendering a list of friends in order to let the user tag them in a post.
If your App is a Game AND your Game supports Facebook Canvas, you can use the /me/invitable_friends endpoint in order to render a custom invite dialog, then pass the tokens returned by this API to the standard Requests Dialog.
In other cases, apps are no longer able to retrieve the full list of a user's friends (only those friends who have specifically authorized your app using the user_friends permission). This has been confirmed by Facebook as 'by design'.
For apps wanting allow people to invite friends to use an app, you can still use the Send Dialog on Web or the new Message Dialog on iOS and Android.
UPDATE: Facebook have published an FAQ on these changes here: https://developers.facebook.com/docs/apps/faq which explain all the options available to developers in order to invite friends etc.
Although Simon Cross's answer is accepted and correct, I thought I would beef it up a bit with an example (Android) of what needs to be done. I'll keep it as general as I can and focus on just the question. Personally I wound up storing things in a database so the loading was smooth, but that requires a CursorAdapter and ContentProvider which is a bit out of scope here.
I came here myself and then thought, now what?!
The Issue
Just like user3594351, I was noticing the friend data was blank. I found this out by using the FriendPickerFragment. What worked three months ago, no longer works. Even Facebook's examples broke. So my issue was 'How Do I create FriendPickerFragment by hand?
What Did Not Work
Option #1 from Simon Cross was not strong enough to invite friends to the app. Simon Cross also recommended the Requests Dialog, but that would only allow five requests at a time. The requests dialog also showed the same friends during any given Facebook logged in session. Not useful.
What Worked (Summary)
Option #2 with some hard work. You must make sure you fulfill Facebook's new rules: 1.) You're a game 2.) You have a Canvas app (Web Presence) 3.) Your app is registered with Facebook. It is all done on the Facebook developer website under Settings.
To emulate the friend picker by hand inside my app I did the following:
Create a tab activity that shows two fragments. Each fragment shows a list. One fragment for available friend (/me/friends) and another for invitable friends (/me/invitable_friends). Use the same fragment code to render both tabs.
Create an AsyncTask that will get the friend data from Facebook. Once that data is loaded, toss it to the adapter which will render the values to the screen.
Details
The AsynchTask
private class DownloadFacebookFriendsTask extends AsyncTask<FacebookFriend.Type, Boolean, Boolean> {
private final String TAG = DownloadFacebookFriendsTask.class.getSimpleName();
GraphObject graphObject;
ArrayList<FacebookFriend> myList = new ArrayList<FacebookFriend>();
#Override
protected Boolean doInBackground(FacebookFriend.Type... pickType) {
//
// Determine Type
//
String facebookRequest;
if (pickType[0] == FacebookFriend.Type.AVAILABLE) {
facebookRequest = "/me/friends";
} else {
facebookRequest = "/me/invitable_friends";
}
//
// Launch Facebook request and WAIT.
//
new Request(
Session.getActiveSession(),
facebookRequest,
null,
HttpMethod.GET,
new Request.Callback() {
public void onCompleted(Response response) {
FacebookRequestError error = response.getError();
if (error != null && response != null) {
Log.e(TAG, error.toString());
} else {
graphObject = response.getGraphObject();
}
}
}
).executeAndWait();
//
// Process Facebook response
//
//
if (graphObject == null) {
return false;
}
int numberOfRecords = 0;
JSONArray dataArray = (JSONArray) graphObject.getProperty("data");
if (dataArray.length() > 0) {
// Ensure the user has at least one friend ...
for (int i = 0; i < dataArray.length(); i++) {
JSONObject jsonObject = dataArray.optJSONObject(i);
FacebookFriend facebookFriend = new FacebookFriend(jsonObject, pickType[0]);
if (facebookFriend.isValid()) {
numberOfRecords++;
myList.add(facebookFriend);
}
}
}
// Make sure there are records to process
if (numberOfRecords > 0){
return true;
} else {
return false;
}
}
#Override
protected void onProgressUpdate(Boolean... booleans) {
// No need to update this, wait until the whole thread finishes.
}
#Override
protected void onPostExecute(Boolean result) {
if (result) {
/*
User the array "myList" to create the adapter which will control showing items in the list.
*/
} else {
Log.i(TAG, "Facebook Thread unable to Get/Parse friend data. Type = " + pickType);
}
}
}
The FacebookFriend class I created
public class FacebookFriend {
String facebookId;
String name;
String pictureUrl;
boolean invitable;
boolean available;
boolean isValid;
public enum Type {AVAILABLE, INVITABLE};
public FacebookFriend(JSONObject jsonObject, Type type) {
//
//Parse the Facebook Data from the JSON object.
//
try {
if (type == Type.INVITABLE) {
//parse /me/invitable_friend
this.facebookId = jsonObject.getString("id");
this.name = jsonObject.getString("name");
// Handle the picture data.
JSONObject pictureJsonObject = jsonObject.getJSONObject("picture").getJSONObject("data");
boolean isSilhouette = pictureJsonObject.getBoolean("is_silhouette");
if (!isSilhouette) {
this.pictureUrl = pictureJsonObject.getString("url");
} else {
this.pictureUrl = "";
}
this.invitable = true;
} else {
// Parse /me/friends
this.facebookId = jsonObject.getString("id");
this.name = jsonObject.getString("name");
this.available = true;
this.pictureUrl = "";
}
isValid = true;
} catch (JSONException e) {
Log.w("#", "Warnings - unable to process Facebook JSON: " + e.getLocalizedMessage());
}
}
}
Facebook has revised their policies now. You can’t get the whole friendlist anyway if your app does not have a Canvas implementation and if your app is not a game. Of course there’s also taggable_friends, but that one is for tagging only.
You will be able to pull the list of friends who have authorised the app only.
The apps that are using Graph API 1.0 will be working till April 30th, 2015 and after that it will be deprecated.
See the following to get more details on this:
User Friends
Facebook Application Development FAQ
In Swift 4.2 and Xcode 10.1:
If you want to get the friends list from Facebook, you need to submit your app for review in Facebook. See some of the Login Permissions:
Login Permissions
Here are the two steps:
1) First your app status is must be in Live
2) Get required permissions form Facebook.
1) Enable our app status live:
Go to the apps page and select your app
https://developers.facebook.com/apps/
Select status in the top right in Dashboard.
Submit privacy policy URL
Select category
Now our app is in Live status.
One step is completed.
2) Submit our app for review:
First send required requests.
Example: user_friends, user_videos, user_posts, etc.
Second, go to the Current Request page
Example: user_events
Submit all details
Like this submit for all requests (user_friends , user_events, user_videos, user_posts, etc.).
Finally submit your app for review.
If your review is accepted from Facebook's side, you are now eligible to read contacts, etc.
As Simon mentioned, this is not possible in the new Facebook API. Pure technically speaking you can do it via browser automation.
this is against Facebook policy, so depending on the country where you live, this may not be legal
you'll have to use your credentials / ask user for credentials and possibly store them (storing passwords even symmetrically encrypted is not a good idea)
when Facebook changes their API, you'll have to update the browser automation code as well (if you can't force updates of your application, you should put browser automation piece out as a webservice)
this is bypassing the OAuth concept
on the other hand, my feeling is that I'm owning my data including the list of my friends and Facebook shouldn't restrict me from accessing those via the API
Sample implementation using WatiN:
class FacebookUser
{
public string Name { get; set; }
public long Id { get; set; }
}
public IList<FacebookUser> GetFacebookFriends(string email, string password, int? maxTimeoutInMilliseconds)
{
var users = new List<FacebookUser>();
Settings.Instance.MakeNewIeInstanceVisible = false;
using (var browser = new IE("https://www.facebook.com"))
{
try
{
browser.TextField(Find.ByName("email")).Value = email;
browser.TextField(Find.ByName("pass")).Value = password;
browser.Form(Find.ById("login_form")).Submit();
browser.WaitForComplete();
}
catch (ElementNotFoundException)
{
// We're already logged in
}
browser.GoTo("https://www.facebook.com/friends");
var watch = new Stopwatch();
watch.Start();
Link previousLastLink = null;
while (maxTimeoutInMilliseconds.HasValue && watch.Elapsed.TotalMilliseconds < maxTimeoutInMilliseconds.Value)
{
var lastLink = browser.Links.Where(l => l.GetAttributeValue("data-hovercard") != null
&& l.GetAttributeValue("data-hovercard").Contains("user.php")
&& l.Text != null
).LastOrDefault();
if (lastLink == null || previousLastLink == lastLink)
{
break;
}
var ieElement = lastLink.NativeElement as IEElement;
if (ieElement != null)
{
var htmlElement = ieElement.AsHtmlElement;
htmlElement.scrollIntoView();
browser.WaitForComplete();
}
previousLastLink = lastLink;
}
var links = browser.Links.Where(l => l.GetAttributeValue("data-hovercard") != null
&& l.GetAttributeValue("data-hovercard").Contains("user.php")
&& l.Text != null
).ToList();
var idRegex = new Regex("id=(?<id>([0-9]+))");
foreach (var link in links)
{
string hovercard = link.GetAttributeValue("data-hovercard");
var match = idRegex.Match(hovercard);
long id = 0;
if (match.Success)
{
id = long.Parse(match.Groups["id"].Value);
}
users.Add(new FacebookUser
{
Name = link.Text,
Id = id
});
}
}
return users;
}
Prototype with implementation of this approach (using C#/WatiN) see https://github.com/svejdo1/ShadowApi. It is also allowing dynamic update of Facebook connector that is retrieving a list of your contacts.
Try /me/taggable_friends?limit=5000 using your JavaScript code
Or
try the Graph API:
https://graph.facebook.com/v2.3/user_id_here/taggable_friends?access_token=
If you are still struggling with this issue on a development mode.
Follow the same process as mentioned below:
create a test app of your main app,
create test users, automatically install app for test users and assign them 'user_friend' permission.
Add your test users as a friend with each other.
I followed the same process after going through alot of research and finally it worked.
In the Facebook SDK Graph API v2.0 or above, you must request the user_friends permission from each user in the time of Facebook login since user_friends is no longer included by default in every login; we have to add that.
Each user must grant the user_friends permission in order to appear in the response to /me/friends.
let fbLoginManager : FBSDKLoginManager = FBSDKLoginManager()
fbLoginManager.loginBehavior = FBSDKLoginBehavior.web
fbLoginManager.logIn(withReadPermissions: ["email","user_friends","public_profile"], from: self) { (result, error) in
if (error == nil) {
let fbloginresult : FBSDKLoginManagerLoginResult = result!
if fbloginresult.grantedPermissions != nil {
if (fbloginresult.grantedPermissions.contains("email")) {
// Do the stuff
}
else {
}
}
else {
}
}
}
So at the time of Facebook login, it prompts with a screen which contain all the permissions:
If the user presses the Continue button, the permissions will be set. When you access the friends list using Graph API, your friends who logged into the application as above will be listed
if ((FBSDKAccessToken.current()) != nil) {
FBSDKGraphRequest(graphPath: "/me/friends", parameters: ["fields" : "id,name"]).start(completionHandler: { (connection, result, error) -> Void in
if (error == nil) {
print(result!)
}
})
}
The output will contain the users who granted the user_friends permission at the time of login to your application through Facebook.
{
data = (
{
id = xxxxxxxxxx;
name = "xxxxxxxx";
}
);
paging = {
cursors = {
after = xxxxxx;
before = xxxxxxx;
};
};
summary = {
"total_count" = 8;
};
}
I am trying to get my friend name and ids with Graph API v2.0, but data returns empty:
{
"data": [
]
}
When I was using v1.0, everything was OK with the following request:
FBRequest* friendsRequest = [FBRequest requestForMyFriends];
[friendsRequest startWithCompletionHandler: ^(FBRequestConnection *connection,
NSDictionary* result,
NSError *error) {
NSArray* friends = [result objectForKey:#"data"];
NSLog(#"Found: %i friends", friends.count);
for (NSDictionary<FBGraphUser>* friend in friends) {
NSLog(#"I have a friend named %# with id %#", friend.name, friend.id);
}
}];
But now I cannot get friends!
In v2.0 of the Graph API, calling /me/friends returns the person's friends who also use the app.
In addition, in v2.0, you must request the user_friends permission from each user. user_friends is no longer included by default in every login. Each user must grant the user_friends permission in order to appear in the response to /me/friends. See the Facebook upgrade guide for more detailed information, or review the summary below.
If you want to access a list of non-app-using friends, there are two options:
If you want to let your people tag their friends in stories that they publish to Facebook using your App, you can use the /me/taggable_friends API. Use of this endpoint requires review by Facebook and should only be used for the case where you're rendering a list of friends in order to let the user tag them in a post.
If your App is a Game AND your Game supports Facebook Canvas, you can use the /me/invitable_friends endpoint in order to render a custom invite dialog, then pass the tokens returned by this API to the standard Requests Dialog.
In other cases, apps are no longer able to retrieve the full list of a user's friends (only those friends who have specifically authorized your app using the user_friends permission). This has been confirmed by Facebook as 'by design'.
For apps wanting allow people to invite friends to use an app, you can still use the Send Dialog on Web or the new Message Dialog on iOS and Android.
UPDATE: Facebook have published an FAQ on these changes here: https://developers.facebook.com/docs/apps/faq which explain all the options available to developers in order to invite friends etc.
Although Simon Cross's answer is accepted and correct, I thought I would beef it up a bit with an example (Android) of what needs to be done. I'll keep it as general as I can and focus on just the question. Personally I wound up storing things in a database so the loading was smooth, but that requires a CursorAdapter and ContentProvider which is a bit out of scope here.
I came here myself and then thought, now what?!
The Issue
Just like user3594351, I was noticing the friend data was blank. I found this out by using the FriendPickerFragment. What worked three months ago, no longer works. Even Facebook's examples broke. So my issue was 'How Do I create FriendPickerFragment by hand?
What Did Not Work
Option #1 from Simon Cross was not strong enough to invite friends to the app. Simon Cross also recommended the Requests Dialog, but that would only allow five requests at a time. The requests dialog also showed the same friends during any given Facebook logged in session. Not useful.
What Worked (Summary)
Option #2 with some hard work. You must make sure you fulfill Facebook's new rules: 1.) You're a game 2.) You have a Canvas app (Web Presence) 3.) Your app is registered with Facebook. It is all done on the Facebook developer website under Settings.
To emulate the friend picker by hand inside my app I did the following:
Create a tab activity that shows two fragments. Each fragment shows a list. One fragment for available friend (/me/friends) and another for invitable friends (/me/invitable_friends). Use the same fragment code to render both tabs.
Create an AsyncTask that will get the friend data from Facebook. Once that data is loaded, toss it to the adapter which will render the values to the screen.
Details
The AsynchTask
private class DownloadFacebookFriendsTask extends AsyncTask<FacebookFriend.Type, Boolean, Boolean> {
private final String TAG = DownloadFacebookFriendsTask.class.getSimpleName();
GraphObject graphObject;
ArrayList<FacebookFriend> myList = new ArrayList<FacebookFriend>();
#Override
protected Boolean doInBackground(FacebookFriend.Type... pickType) {
//
// Determine Type
//
String facebookRequest;
if (pickType[0] == FacebookFriend.Type.AVAILABLE) {
facebookRequest = "/me/friends";
} else {
facebookRequest = "/me/invitable_friends";
}
//
// Launch Facebook request and WAIT.
//
new Request(
Session.getActiveSession(),
facebookRequest,
null,
HttpMethod.GET,
new Request.Callback() {
public void onCompleted(Response response) {
FacebookRequestError error = response.getError();
if (error != null && response != null) {
Log.e(TAG, error.toString());
} else {
graphObject = response.getGraphObject();
}
}
}
).executeAndWait();
//
// Process Facebook response
//
//
if (graphObject == null) {
return false;
}
int numberOfRecords = 0;
JSONArray dataArray = (JSONArray) graphObject.getProperty("data");
if (dataArray.length() > 0) {
// Ensure the user has at least one friend ...
for (int i = 0; i < dataArray.length(); i++) {
JSONObject jsonObject = dataArray.optJSONObject(i);
FacebookFriend facebookFriend = new FacebookFriend(jsonObject, pickType[0]);
if (facebookFriend.isValid()) {
numberOfRecords++;
myList.add(facebookFriend);
}
}
}
// Make sure there are records to process
if (numberOfRecords > 0){
return true;
} else {
return false;
}
}
#Override
protected void onProgressUpdate(Boolean... booleans) {
// No need to update this, wait until the whole thread finishes.
}
#Override
protected void onPostExecute(Boolean result) {
if (result) {
/*
User the array "myList" to create the adapter which will control showing items in the list.
*/
} else {
Log.i(TAG, "Facebook Thread unable to Get/Parse friend data. Type = " + pickType);
}
}
}
The FacebookFriend class I created
public class FacebookFriend {
String facebookId;
String name;
String pictureUrl;
boolean invitable;
boolean available;
boolean isValid;
public enum Type {AVAILABLE, INVITABLE};
public FacebookFriend(JSONObject jsonObject, Type type) {
//
//Parse the Facebook Data from the JSON object.
//
try {
if (type == Type.INVITABLE) {
//parse /me/invitable_friend
this.facebookId = jsonObject.getString("id");
this.name = jsonObject.getString("name");
// Handle the picture data.
JSONObject pictureJsonObject = jsonObject.getJSONObject("picture").getJSONObject("data");
boolean isSilhouette = pictureJsonObject.getBoolean("is_silhouette");
if (!isSilhouette) {
this.pictureUrl = pictureJsonObject.getString("url");
} else {
this.pictureUrl = "";
}
this.invitable = true;
} else {
// Parse /me/friends
this.facebookId = jsonObject.getString("id");
this.name = jsonObject.getString("name");
this.available = true;
this.pictureUrl = "";
}
isValid = true;
} catch (JSONException e) {
Log.w("#", "Warnings - unable to process Facebook JSON: " + e.getLocalizedMessage());
}
}
}
Facebook has revised their policies now. You can’t get the whole friendlist anyway if your app does not have a Canvas implementation and if your app is not a game. Of course there’s also taggable_friends, but that one is for tagging only.
You will be able to pull the list of friends who have authorised the app only.
The apps that are using Graph API 1.0 will be working till April 30th, 2015 and after that it will be deprecated.
See the following to get more details on this:
User Friends
Facebook Application Development FAQ
In Swift 4.2 and Xcode 10.1:
If you want to get the friends list from Facebook, you need to submit your app for review in Facebook. See some of the Login Permissions:
Login Permissions
Here are the two steps:
1) First your app status is must be in Live
2) Get required permissions form Facebook.
1) Enable our app status live:
Go to the apps page and select your app
https://developers.facebook.com/apps/
Select status in the top right in Dashboard.
Submit privacy policy URL
Select category
Now our app is in Live status.
One step is completed.
2) Submit our app for review:
First send required requests.
Example: user_friends, user_videos, user_posts, etc.
Second, go to the Current Request page
Example: user_events
Submit all details
Like this submit for all requests (user_friends , user_events, user_videos, user_posts, etc.).
Finally submit your app for review.
If your review is accepted from Facebook's side, you are now eligible to read contacts, etc.
As Simon mentioned, this is not possible in the new Facebook API. Pure technically speaking you can do it via browser automation.
this is against Facebook policy, so depending on the country where you live, this may not be legal
you'll have to use your credentials / ask user for credentials and possibly store them (storing passwords even symmetrically encrypted is not a good idea)
when Facebook changes their API, you'll have to update the browser automation code as well (if you can't force updates of your application, you should put browser automation piece out as a webservice)
this is bypassing the OAuth concept
on the other hand, my feeling is that I'm owning my data including the list of my friends and Facebook shouldn't restrict me from accessing those via the API
Sample implementation using WatiN:
class FacebookUser
{
public string Name { get; set; }
public long Id { get; set; }
}
public IList<FacebookUser> GetFacebookFriends(string email, string password, int? maxTimeoutInMilliseconds)
{
var users = new List<FacebookUser>();
Settings.Instance.MakeNewIeInstanceVisible = false;
using (var browser = new IE("https://www.facebook.com"))
{
try
{
browser.TextField(Find.ByName("email")).Value = email;
browser.TextField(Find.ByName("pass")).Value = password;
browser.Form(Find.ById("login_form")).Submit();
browser.WaitForComplete();
}
catch (ElementNotFoundException)
{
// We're already logged in
}
browser.GoTo("https://www.facebook.com/friends");
var watch = new Stopwatch();
watch.Start();
Link previousLastLink = null;
while (maxTimeoutInMilliseconds.HasValue && watch.Elapsed.TotalMilliseconds < maxTimeoutInMilliseconds.Value)
{
var lastLink = browser.Links.Where(l => l.GetAttributeValue("data-hovercard") != null
&& l.GetAttributeValue("data-hovercard").Contains("user.php")
&& l.Text != null
).LastOrDefault();
if (lastLink == null || previousLastLink == lastLink)
{
break;
}
var ieElement = lastLink.NativeElement as IEElement;
if (ieElement != null)
{
var htmlElement = ieElement.AsHtmlElement;
htmlElement.scrollIntoView();
browser.WaitForComplete();
}
previousLastLink = lastLink;
}
var links = browser.Links.Where(l => l.GetAttributeValue("data-hovercard") != null
&& l.GetAttributeValue("data-hovercard").Contains("user.php")
&& l.Text != null
).ToList();
var idRegex = new Regex("id=(?<id>([0-9]+))");
foreach (var link in links)
{
string hovercard = link.GetAttributeValue("data-hovercard");
var match = idRegex.Match(hovercard);
long id = 0;
if (match.Success)
{
id = long.Parse(match.Groups["id"].Value);
}
users.Add(new FacebookUser
{
Name = link.Text,
Id = id
});
}
}
return users;
}
Prototype with implementation of this approach (using C#/WatiN) see https://github.com/svejdo1/ShadowApi. It is also allowing dynamic update of Facebook connector that is retrieving a list of your contacts.
Try /me/taggable_friends?limit=5000 using your JavaScript code
Or
try the Graph API:
https://graph.facebook.com/v2.3/user_id_here/taggable_friends?access_token=
If you are still struggling with this issue on a development mode.
Follow the same process as mentioned below:
create a test app of your main app,
create test users, automatically install app for test users and assign them 'user_friend' permission.
Add your test users as a friend with each other.
I followed the same process after going through alot of research and finally it worked.
In the Facebook SDK Graph API v2.0 or above, you must request the user_friends permission from each user in the time of Facebook login since user_friends is no longer included by default in every login; we have to add that.
Each user must grant the user_friends permission in order to appear in the response to /me/friends.
let fbLoginManager : FBSDKLoginManager = FBSDKLoginManager()
fbLoginManager.loginBehavior = FBSDKLoginBehavior.web
fbLoginManager.logIn(withReadPermissions: ["email","user_friends","public_profile"], from: self) { (result, error) in
if (error == nil) {
let fbloginresult : FBSDKLoginManagerLoginResult = result!
if fbloginresult.grantedPermissions != nil {
if (fbloginresult.grantedPermissions.contains("email")) {
// Do the stuff
}
else {
}
}
else {
}
}
}
So at the time of Facebook login, it prompts with a screen which contain all the permissions:
If the user presses the Continue button, the permissions will be set. When you access the friends list using Graph API, your friends who logged into the application as above will be listed
if ((FBSDKAccessToken.current()) != nil) {
FBSDKGraphRequest(graphPath: "/me/friends", parameters: ["fields" : "id,name"]).start(completionHandler: { (connection, result, error) -> Void in
if (error == nil) {
print(result!)
}
})
}
The output will contain the users who granted the user_friends permission at the time of login to your application through Facebook.
{
data = (
{
id = xxxxxxxxxx;
name = "xxxxxxxx";
}
);
paging = {
cursors = {
after = xxxxxx;
before = xxxxxxx;
};
};
summary = {
"total_count" = 8;
};
}
I have 2 services that are defined in the same thrift file and share a port. I can use any method from serviceA no problem but whenever i try to call any of ServiceB's methods i get the exception.
this is my thrift file (service-a.thrift):
service ServiceA extends common.CommonService {
list<i64> getByIds(1: list<i64> ids)
...
}
service ServiceB extends common.CommonService {
list<i64> getByIds(1: list<i64> ids)
...
}
notes:
I'm working with a python client
Thrift version 0.8.0
Any ideas?
We had this need as well and solved by writing a new implementation of TProcessor that creates a map of multiple processors. The only gotcha is that with this implementation you need to ensure no method names overlap - i.e. don't use nice generic names like Run() in different servers. Apologies on not converting C# to Python...
Example Class:
using System;
using System.Collections;
using System.Collections.Generic;
using System.IO;
using System.Reflection;
using Thrift;
using Thrift.Protocol;
/// <summary>
/// Processor that allows for multiple services to run under one roof. Requires no method name conflicts across services.
/// </summary>
public class MultiplexProcessor : TProcessor {
public MultiplexProcessor(IEnumerable<TProcessor> processors) {
ProcessorMap = new Dictionary<string, Tuple<TProcessor, Delegate>>();
foreach (var processor in processors) {
var processMap = (IDictionary) processor.GetType().GetField("processMap_", BindingFlags.NonPublic | BindingFlags.Instance).GetValue(processor);
foreach (string pmk in processMap.Keys) {
var imp = (Delegate) processMap[pmk];
try {
ProcessorMap.Add(pmk, new Tuple<TProcessor, Delegate>(processor, imp));
}
catch (ArgumentException) {
throw new ArgumentException(string.Format("Method already exists in process map: {0}", pmk));
}
}
}
}
protected readonly Dictionary<string, Tuple<TProcessor, Delegate>> ProcessorMap;
internal protected Dictionary<string, Tuple<TProcessor, Delegate>> GetProcessorMap() {
return new Dictionary<string, Tuple<TProcessor, Delegate>>(ProcessorMap);
}
public bool Process(TProtocol iprot, TProtocol oprot) {
try {
TMessage msg = iprot.ReadMessageBegin();
Tuple<TProcessor, Delegate> fn;
ProcessorMap.TryGetValue(msg.Name, out fn);
if (fn == null) {
TProtocolUtil.Skip(iprot, TType.Struct);
iprot.ReadMessageEnd();
var x = new TApplicationException(TApplicationException.ExceptionType.UnknownMethod, "Invalid method name: '" + msg.Name + "'");
oprot.WriteMessageBegin(new TMessage(msg.Name, TMessageType.Exception, msg.SeqID));
x.Write(oprot);
oprot.WriteMessageEnd();
oprot.Transport.Flush();
return true;
}
Console.WriteLine("Invoking service method {0}.{1}", fn.Item1, fn.Item2);
fn.Item2.Method.Invoke(fn.Item1, new object[] {msg.SeqID, iprot, oprot});
}
catch (IOException) {
return false;
}
return true;
}
}
Example Usage:
Processor = new MultiplexProcessor(
new List<TProcessor> {
new ReportingService.Processor(new ReportingServer()),
new MetadataService.Processor(new MetadataServer()),
new OtherService.Processor(new OtherService())
}
);
Server = new TThreadPoolServer(Processor, Transport);
As far as I know, there's no straightforward way to bind several services to a single port without adding this field to TMessage and recompiling thrift. If you want to have two services using the same Server, you should reimplement Thrift Server, which it doesn't seem an easy task