Develop Reference

Scraping Yelp review content displaying different tags using Beautiful Soup

I'm practicing web-scraping and trying to grab the reviews from the following page: https://www.yelp.com/biz/jajaja-plantas-mexicana-new-york-2?osq=Vegetarian+Food
This is what I have so far after inspecting the name element on the webpage:
page = requests.get('https://www.yelp.com/biz/jajaja-plantas-mexicana-new-york-2?osq=Vegetarian+Food', headers={'User-Agent':'Mozilla/5.0'}).text
parsed_page = BeautifulSoup(page, 'lxml')
# print(parsed_page)
for x in parsed_page.find_all('a', class_='css-1422juy'):
print(x)
But it doesn't seem to be working, the output is not the name but:
<a class="css-1422juy" href="/c/mexican">Mexican</a>
<a class="css-1422juy" href="/c/vegan">Vegan</a>
<a class="css-1422juy" href="/c/bars">Bars</a>
<a class="css-1422juy" href="https://www.yelp.com/menu/jajaja-plantas-mexicana-new-york-2" role="link">View full menu<span class="display--inline__09f24__c6N_k margin-l1__09f24__m8GL9 border-color--default__09f24__NPAKY"><span aria-hidden="true" class="icon--14-chevron-right-outline css-mpwjkl"><svg class="icon_svg" height="14" width="14"><path d="M5.043 11.5a.498.498 0 00.353-.146L9.75 7 5.396 2.646a.5.5 0 00-.707.708L8.336 7l-3.647 3.646a.502.502 0 00.354.854z"></path></svg></span></span></a>
<a class="css-1422juy" href="/questions/XipQLDbyTl5tsLlyzAWzug" role="link">Ask a question<span class="display--inline__09f24__c6N_k margin-l1__09f24__m8GL9 border-color--default__09f24__NPAKY"><span aria-hidden="true" class="icon--24-add-v2 css-106vfgv"><svg class="icon_svg" height="24" width="24"><path d="M19 11h-6V5a1 1 0 10-2 0v6H5a1 1 0 100 2h6v6a1 1 0 102 0v-6h6a1 1 0 100-2z"></path></svg></span></span></a>
If I use html.parser and .prettify() instead the parsed output of the name, rating, and review fields in the the console looks pretty different:
<script type="application/ld+json">
{"#context":"https://schema.org","#type":"Restaurant","name":"Jajaja Plantas Mexicana","image":"https://s3-media0.fl.yelpcdn.com/bphoto/OkWKXxOZBLJO7hRjOlIMig/l.jpg","priceRange":"$11-30","telephone":"(646) 883-5453","address":{"streetAddress":"162 E Broadway","addressLocality":"New York","addressCountry":"US","addressRegion":"NY","postalCode":"10002"},"review":[{"author":"Caroline J.","datePublished":"2021-11-24","reviewRating":{"ratingValue":5},"description":"I typically don&apos;t write reviews for restaurants, but I&apos;ll make an exception! I have been vegan for almost 4 years now and meatless for about 8 years now. My sister told me about this place and we didn&apos;t get a chance to go until last night. My sister is non-vegan, my mom is also non-vegan. I went with my sister, and we got food to share. My sister who is a very skeptical about vegan food absolutely loved it. My mom who is also skeptical about vegan food had my sisters leftovers and absolutely loved it as well! Everything was delicious and I love the atmosphere in the restaurant as well! Definitely will be going back because they have other stuff on the menu I want to try! It&apos;s both vegan and non vegan approved for a 100% vegan Mexican restaurant!!"},{"author":"Melissa W.","datePublished":"2021-12-24","reviewRating":{"ratingValue":5},"description":"The food was great. I went with my partner for a birthday dinner and while I have adopted a plant based lifestyle, he has not but he enjoyed EVERYTHING that we ordered."},{"author":"Sooji L.","datePublished":"2022-01-09","reviewRating":{"ratingValue":5},"description":"NACHOS NACHOS NACHOS ($18 with guac). I prefer having these nachos than other nachos that aren&apos;t vegan. Soooooooo gooooood. I have to say that to pay extra $3 for guac, they can add more than just a scoop. The portions are enough for 3 people! \n\nFish tacos were great. Replaced with squash and flavorful all together. $9 for two tacos. You may add an extra taco for an extra $3/4 bucks. \n\nMy husband ordered the Gorditas. The shell was so crunchy and good. It was supposed to have "bacon" but it was quite underwhelming. Wouldn&apos;t quite recommend this dish.\n\nBoth drinks were delish! Almond horchata... I would get that the next time I come back. It is a bit gritty though.\nI ordered the matcha y coconut cocktail. Cocktails are $$. It was $15. Some drinks can be turned into a mock tail for the same price which I find that to be unreasonable. Regardless, the drinks are worth getting.\n\n****ASK FOR HOT SAUCE! and you&apos;ll get three choices. I love my hot sauce and the variety they provided. The mild (orange) was my favorite. \n\n(Rating was not based on this following experience) on our visit, there was a homeless man in their outdoor seating area which we were able to view from our seat inside. At some moments, he pulled down his pants. Not pleasant, I&apos;d say. The staff did try to remove him but he did not budge. I hope they reached out to homeless services. He was there during our entire dining experience. \n\nMy rating is purely on the food and service. That situation was out of their control but do please keep in mind that the neighborhood is not the best area. They do have multiple locations so please consider checking them out."},{"author":"Roshni P.","datePublished":"2021-11-11","reviewRating":{"ratingValue":5},"description":"I was skeptical because I am from Southern California and Mexican has to hit right for me. But holy crap this is amazing!!!! \n\nBirria tacos was our least favorite but that&apos;s not saying much because I would give it 9.5/10 \n\nBurrito was 10/10 . As you eat it there is more and more flavor and so so good! I like the red sauce side more \n\nNachos is hands down the best item!!! Plenty of food for your meal. I would give it a 11/10. The chorizo is soo good. Ask for hot sauces on the side. The orange one is really good"},{"author":"Whitney L.","datePublished":"2021-11-03","reviewRating":{"ratingValue":5},"description":"This place hits the spot on so many levels.\n\n Service was attentive, efficient and personable. Check!\n\nFood---outstanding. I ordered the Tavon Taco bowl, pumpkin and beet empanada and the Matcha and Coconut Cocktail. Everything was delicious and seasoned well. If you like matcha, don&apos;t sleep on that Matcha +Coconut Cocktail....OUTSTANDING. Everything tasted truly unique---like something you can&apos;t get other places. And even better, lots of veggies and healthy options. Double- check!!\n\nTheir homemade chorizo is yum yum yum. Make sure to try that!\n\nDecor and ambiance was fun, colorful and on brand. This place has great energy. Highly recommend and can&apos;t wait to go back."},{"author":"Johnny G.","datePublished":"2021-12-27","reviewRating":{"ratingValue":4},"description":"The nachos are to die for. Unbelievable, so clean and not heavy. Couldn&apos;t tell the difference from any carnivorous nacho I&apos;ve had. I loved it and wished I stopped there.\nI had the Coconut Queso Quesadilla and wasn&apos;t really a fan. Tasted like a pasta dish (heavy on the pesto) and it had a sweetness. Nonetheless great food overall."},{"author":"Kathy X.","datePublished":"2021-09-25","reviewRating":{"ratingValue":4},"description":"Recently, I came here with a friend for lunch. I was looking forward to trying plant-based food and drink. The interior was lovely, bright, and modern. There is a fully stocked bar with plants all around, orange lights dropping down from the ceiling, and nice wood tables. \n\nFor my meal, I ordered Almond Horchata (always have to get the Horchata if it&apos;s on the menu), Crispy Chayote Fish, and Enchiladas Mole. I also got to try my friend&apos;s Chorizo Nachos and Mexican Street Corn. \n\nThe Almond Horchata came in a wine glass with a beautiful deep purple orchid flower. The color of the drink was pale brown with bits of coconut on top. It was delightfully creamy, cinnamon-y, and sweet. \n\nChayote Fish consisted of hemp and flax seed buttered squash, and the taco also came with chipotle almond butter and red onion. The squash was fried to perfection in a deliciously well-seasoned batter, and truly resembled the taste of fish. The chipotle-flavored butter added a depth of flavor to the tacos, while the red onion gave it an additional crunch and fresh element. \n\nEnchiladas Moles had shredded palm carnitas, coconut queso, guajillo, sour cream, and also came with Spanish rice. This was absolutely wonderful! The palm carnitas was tender, succulent, with plenty of flavorful spices. The sauce that accompanied the enchiladas was yummy as well. In addition, the Spanish rice was cooked well. \n\nNachos came with plant based chorizo, fermented black beans, corn, turmeric, queso fundido, and sour cream. What a mouthwatering medley of flavors that mingled together to create very tasty bites. The chorizo was juicy, succulent, and packed with bold flavor. The cheese sauce with the sour cream was addicting and was well distributed throughout all the chips. \n\nThe Mexican Street Corn was covered with a powdery type of cheese, and slightly charred (which is how I like it) and was good. \n\nService was top notch. Our waiter was friendly and kind. Everything was made with such care and attention to detail. I also loved the unique menu offerings. I would definitely like to return and try the Coconut Queso Quesadilla, and perhaps the Peanut Chocolate Cake."},{"author":"Alissa M.","datePublished":"2022-01-09","reviewRating":{"ratingValue":4},"description":"Came here for lunch with my wife on a Saturday afternoon. The restaurant is absolutely adorable inside! I love the plants and the decor all throughout. The nachos are to die for! Definitely the best nachos I have had since becoming dairy free. The coquito was also very good! The display of the chorizo burrito and barbacoa tacos was cute but I wasn&apos;t a huge fan of the texture of the barbacoa or the flavor of the protein inside the burrito. We will definitely be back for the Nachos and I must try the mini churros next time!"},{"author":"Steven Z.","datePublished":"2021-08-12","reviewRating":{"ratingValue":5},"description":"Great Mexican food with a cool interior playing Hispanic music which made for a nice ambience. Prices are fair\n\nMy friend and I were starving after a long day and ordered a lot of food: nachos, chorizo burrito, fish tacos and a quesadilla. \n\nThe nachos (with guac) were really good and made for a great appetizer, the chorizo burrito was delicious (but very messy since it&apos;s covered in sauce), the quesadilla was really good too. But the highlight of the meal was the fish tacos - nice crispy outside but soft and yummy inside. If I had to get one dish again, it&apos;d be the fish taco. \n\nThe frozen agave margarita was a refreshing as well; it&apos;s pretty strong so you&apos;ll definitely taste the alcohol."},{"author":"Lily C.","datePublished":"2021-10-30","reviewRating":{"ratingValue":5},"description":"We got the fish tacos, carnitas, nachos with guacamole, and churros! Everything was so good and we couldn&apos;t believe it was all vegan! My boyfriend and I are both meat eaters, but we noticed that we didn&apos;t feel sluggish/sleepy (as we would with our non-vegan meals) after finishing our food. Definitely get the churros if you&apos;re craving for something sweet after your meal! \n\n5/5 Highly recommend whether you&apos;re a vegan or not! :D"},{"author":"Rebecca R.","datePublished":"2021-12-26","reviewRating":{"ratingValue":5},"description":"Yummy food. Inventive and interesting. Wide variety. The nachos were great. Good sized portions. Accommodating for preferences and such. Even the kiddos loved it!"},{"author":"Shubhi M.","datePublished":"2021-07-18","reviewRating":{"ratingValue":5},"description":"I&apos;ve never tried a vegan mexican food that tasted this good. It&apos;s crazy some of the vegan taco options they had (birria, carnitas, chorizo, barbacoa, and fish), I have never had an opportunity to try vegan version of those dishes. I&apos;ve never had barbacoa or fish but I can imagine this is may somewhat taste like. It was almost a weird (but definitely good) experience how meaty the texture was of the tacos. \n\nThe nachos were beyond amazing, a pet peeve of mine is when I get served a bunch of chips with toppings only on the top and the bottom of the pile is just plain chips but these were perfectly layered with so much vegan chorizo. They recently opened a location in a whole food which is very close to me so i&apos;m definitely going to go back!"},{"author":"Likitha M.","datePublished":"2021-08-11","reviewRating":{"ratingValue":4},"description":"My friend picked out this place for us to try and I didn&apos;t realize that it was all plant-based until our waitress mentioned it. We didn&apos;t have a reservation but we were seated as soon as we walked in and given a QR code to scan for the menus. \n\nIt felt like there were so many options to choose from but we ultimately narrowed it down and ordered the nachos with added guac, chayote "fish" tacos, chorizo tacos, barbacoa tacos, and almond horchata. We LOVED the nachos -- the chips were nice and crispy and all the toppings were so cohesive. There was a really good topping/chip ratio so we never ran out of one or the other. In addition, the sauces added were delicious -- if you come by and only want one thing to order, get the nachos!! The tacos were all so flavorful and delicious as well, but so small and expensive. I personally don&apos;t think they were worth $8 for 2 tacos, especially if it only took like 3 bites to consume but I understand the pricing as it is vegan and located in NYC. Finally, the horchata was very refreshing and I loved it! \n\nOverall, they had really delicious food and it was a pretty decent experience for my first time having an all plant-based meal!"},{"author":"Fionna L.","datePublished":"2021-07-14","reviewRating":{"ratingValue":4},"description":"Link to menu: https://qrcgcustomers.s3-eu-west-1.amazonaws.com/account8682491/15091203_1.pdf?0.4352589218767642 \n\n(4.5/5) The lively, communal atmosphere made eating here a pleasure! We sat at the bar so service was slower but never neglectful. Their nachos and tacos are must-trys and great to share with friends!\n\n - Nachos: $13 + $3 guac: the BEST plant-based nachos I&apos;ve had! Their chorizo was chewy and flavorful, the fermented black beans were yum and very filling, and the guac was definitely worth it for added depth and creaminess. Small thing, but I wish they had more than just a couple red and purple nachos.\n - Chorizo Burrito: $13: too salty! We could barely taste the ingredients or differentiate the sauces because salt dominated the flavor of every bite. Quite disappointed in this one. \n - Crispy Chayote Fish Tacos (2): $8: how ingenious to use "hemp & flax seed battered squash" as a substitute for fish! The description sounds sophisticated but it tasted very much like BBQ Lay&apos;s, even down to the delicious crunch. Due to the strength of its flavorings, we couldn&apos;t taste any of the squash&apos;s original flavor. The ring of red onion was aesthetic but would have been easier to eat if chopped.\n\nSide note: their restrooms downstairs demonstrate how they turn rustic elements unique through modern and innovative design. \n\nAll in all a great experience, though I wouldn&apos;t recommend the burrito. Would love to go back to try the Coconut Queso Quesadilla and Spicy Birria Tacos!!"},{"author":"Sharon J.","datePublished":"2021-10-16","reviewRating":{"ratingValue":4},"description":"Kinda overrated food, but great vibes! Pretty good for a vegan restaurant, but I&apos;ve had better vegan food and Mexican food. \n\nI got the chorizo burrito but it was very salty with not a lot of other flavor besides that - almost as if they were using salt to cover up the lack of flavor. Definitely very pretty food though, nice interior, and great service - just wish the food was more flavorful. Maybe I&apos;ll try the tacos next time and see if those are better..."},{"author":"Srini V.","datePublished":"2021-09-24","reviewRating":{"ratingValue":5},"description":"I have been meaning to visit this location of Jajaja for some time now. And the opportunity presented itself as a few of us were looking for a neighborhood restaurant in the middle of a torrential downpour.\n\nWe chose to take a table inside. The QR code for their menu was pasted on the wall by our table. We had the guacamole y chips to start. I then got the sopa del día, which happened to be a green soup (sopa verde). This was the highlight of my meal. The soup was thick, had a generous sprinkling of hot sauce and was just perfect, especially after the rain. I then had the chipotle sweet potato street tacos that were delicious ... the fermented beans made for a hearty filling.\n\nGreat service. Nice ambiance. Everyone was happy with their food and drinks. I will be back soon!"},{"author":"Julianna Y.","datePublished":"2021-09-29","reviewRating":{"ratingValue":5},"description":"This place is DELICIOUS! I&apos;ve been ordering my lunch here for a while now, and I have to say that everything I&apos;ve had has been so good! This is a vegan Mexican heaven for foodies! Things you should order when you come here are: \n\n-Turmeric Cauliflower Rice \n-Taco Tazón bowl \n-Nachos! \n-chorizo Burrito \n\nAside from these yummy dishes, their hot sauces are amazing! Definitely recommend getting the mild (orange) and black bean sauce!"},{"author":"Saasha G.","datePublished":"2021-09-14","reviewRating":{"ratingValue":4},"description":"I have became a certified vegan food lover! Never have I tasted such great food and no meat in anything. I was astonished at the flavors. This was my first time dining here and I will return. The place is small and minimal decor but honestly I don&apos;t care just give me the food!"},{"author":"Edward K.","datePublished":"2021-11-20","reviewRating":{"ratingValue":5},"description":"My friend and I get dinner once a month, we alternate choosing and this month my friend selected Jajaja. We arrived a half hour before our 6:45 reservation on a Thursday. The gentleman at the door was able to get us seated right away.\n\nThe restaurant has a trendy vibe, cool furniture and lighting. We did get wedged into a slightly uncomfortable corner table but it may have been uncomfortable mainly because I&apos;m on the taller side.\n\nThe goal of our monthly outings is to continually challenge our palates. We&apos;ve had the vegan/vegetarian genre before but we hadn&apos;t tried vegan Mexican before. The menu is cool, lots of choices that are able to be shared. We selected 5 starters, 2 entree style dishes, and three tacos. The heart of palm ceviche was very fresh. The beet and pumpkin empanada was piping hot and flavorful. I enjoyed the tamale with jackfruit. The Gordita and Burrito entrees were pretty substantial and also tasty. We found the tacos to be a highlight, the cauliflower especially. The birria and "fish" tacos were also quite nice, the birria was perhaps the dish with the most kick and the "fish" had a good texture.\n\nService was solid throughout the meal. Although we ate a solid amount of dishes for two people, I didn&apos;t feel overstuffed. Definitely a place worth returning to."},{"author":"Pamela L.","datePublished":"2021-02-15","reviewRating":{"ratingValue":4},"description":"As I was walking to my mom&apos;s place, this new restaurant stopped me in my tracks. OMG...I remember that this spot used to be a Chinese bakery. As a kid, I used to frequent the old Golden Carriage to grab a drink after a day at the park with my best friend. When I got to my mom&apos;s I check the place out on Yelp. Asked my son if he would be interested in trying Vegan food. He frowned until I convinced him by showing him the photos.\n\nNYC just opened their indoor dining again on the 14th. Temps were taken before we were allowed to sit down. Right off the bat, let me just say that our waiter was amazing, very attentive and kept asking how we liked everything. Decor-wise...I can&apos;t believe this is the same place, it&apos;s an amazing renovation.\n\nOne thing I read and saw was that their nachos w/chorizo is a must order. When it came to the table, I was like...no way we will finish that. Ummm, it was so frickin delish that we had no trouble. The first thing my son said was...I can&apos;t tell that it&apos;s Vegan and he&apos;s a huge meat eater. Our waiter asked if we wanted hot sauce with our nachos. He came by with 3 different ones and explained the level of hottest. The most mild one reminded me of a mole. I thought that was interesting.\n\nWe all enjoyed the tacos we ordered. The portions were generous and the ingredients fresh. I&apos;m not familiar with Vegan cuisine but now I&apos;m game to venture more into it. \n\nOn a last note: there is major gentrification that has been happening in the Lower East Side of Manhattan for years and I for one do welcome it. Some might argue about it but I think it makes the neighborhood more upscale."}],"aggregateRating":{"#type":"AggregateRating","ratingValue":4.5,"reviewCount":1078},"servesCuisine":"Mexican"}
</script>
How would I access something in this particular script tag and the json info inside of it using BeautifulSoup? Or is there a way to do it via the <a> tag with lxml and .find_all()?

You could use json module to parse content of script tags, which is accessible by .text field
Here is the example of parsing all script jsons and printing name:
import json
import requests
from bs4 import BeautifulSoup
url = 'https://www.yelp.com/biz/jajaja-plantas-mexicana-new-york-2?osq=Vegetarian+Food'
r = requests.get(
url,
headers={'User-Agent':'Mozilla/5.0'},
)
parsed_page = BeautifulSoup(r.text, 'html.parser')
page_jsons = []
for x in parsed_page.select('script[type="application/ld+json"]'):
try:
data = json.loads(x.text)
except:
continue
page_jsons.append(data)
for d in page_jsons:
if d.get('name'):
print(d['name']) # => Jajaja Plantas Mexicana

Transforming sentences to Numbers using SciKit-Learn’s CountVectorizer()

I am trying to convert a input sentence Review into a CountVectorizer. I am struggling to handle the sentences that are passed through. How do I deal with the sentences and add vectors to these? Any assistance will be highly appreciated.
Input Data:
Sentiment Review
Neg The new Ford Focus came highly recommended to me when I was looking to buy my first new car I researched its history and found that it received great reviews for comfort and safety during its European release Test driving the car I found it to be comfortable well equipped and stylish I have now driven the car for for 6 months and have put only 5000 miles on it While I have been happy with the overall performance of the car I have been sorely disappointed with the workmanship involved Realizing that new models are notorious for having manufacturing bugs I felt somewhat reassured that these would have been worked out from 1998 1999 during the first European release I was wrong My car has been in the repair shop a total of five times for manufacturers defects including a flooded passenger compartment repaired twice to date faulty master clutch cylinder misaligned striker plate on seat back latch broken break switch and cruise control While I really love my car I would hesitate to recommend it to any but my worst enemies Time will tell if the problems my Focus has had are unique or are related to intrinsic design flaws
Neg We bought the Focus ZTS sedan because my wife needed an economical car to haul the grandkids around with We traded in a 94 Explorer with a 5 speed just before the Firestone tire fiasco became public My wife loves driving the car Although it is a bit small for me 6 1 290lbs it is OK The car handles great and with the Zetec engine it has adequate performance although I wouldnt want any less go than its got Now for the problems the main one of which is because I do my own oil changes A particular sore point for me with most cars is that the manufacturers dont make it easy to change the oil and filter without creating a mess This new Focus is particularly bad First the owners manual indicates a Motocraft FL2005 filter The car had an FL801 on it which some ham fisted factory idiot had torqued to about a million foot pounds I had to use some very large pliers and turn the filter almost 3 4 turn before it was loose enough to move by hand Poor quality control The filter happens to be mounted in a horizontal position and is almost flush with the side of the engine When I finally got it loose oil ran down the side of the engine onto the drive axle onto the frame down my arm and all over the driveway Very bad design On other cars I have been able to use a cut off soda bottle placed over the filter to catch the drips On the Focus it wont work The hood on this car is aluminum It bends very easy mine already has a dent in it and I didnt have an accident A minor problem is the power windows They wont operate with the key in the accessory position Tilt wheel also difficult to operate Bottom line only 3 000 miles on this car but its going to get traded off as soon as possible for a vehicle with a little more substance and which is easier to maintain ive owned 9 Fords since 1986 still have 3 If all the newer Fords are made this way the Focus may be the last Ford product I buy
Neg Recently I had the need to rent a car I picked the Ford Focus I was amazed with this car I liked it better than my own more expensive 1999 Toyota Corolla LE The steering wheel is not only height adjustable but also telescopes something you do not normally find on such a reasonably priced car The drivers seat also adjusted forward and back and in height nice feature for someone tall like myself The front seats were roomy and comfortable and the back seat had I think the most leg room I HAVE EVER SEEN in a compact car The stereo sounded good considering it was stock and the face of the radio has an upward tilt to it so that it is driver friendly All the bells and whistles were located within easy reach and the air worked well In addition to having a roomy trunk there are 60 40 split rear seats Child safety seat anchors and shoulder harness seat belts for 5 passengers I rented the 4 door sedan but there are 3 body styles The 4 door sedan 4 door wagon and a sporty little hatch back I have read the safety ratings for the hatch back and from what I recall it got 5 stars This car is definately on my list of cars to consider purchasing in the near future you should take a look at it too
Pos Cruising In My Big T I have had my 91 Thunderbird for 4 years now bought it way back in my freshman year and it has served me well throughout college I am a horrible Northern driver and brutal on my vehicles but this piece of Ford craftsmanship refuses to bail out on me Its a rough and tumble vehicle that remains an incredible deal for the price especially when bought used from a reputable dealer The Advantages 1 Seat Space These are big seats people with the kind of legroom that only those pretentious you know whats in first class usually get their hands on And that spaciousness isnt just about spoiling the people up front either it extends to the back seat as well which means that everyone feels just a little bit more comfortable and relaxed when you get to wherever you re going And not only are the seats big but the generous amount of padding in each makes for an especially comfortable ride 2 Appearance ive gt to admit it to you I just love the look of the Thunderbird though it is an acquired taste to be sure I can best describe the style as Italian sleek in a chunky way and available in colors like burgundy that make it look like a cross between a hit mobile and a hearse 3 Smooth Ride Riding in my Thunderbird has always seemed quite smooth to me especially when you consider how low to the ground it is Why so low That kind of positioning allows the Thunderbird to provide the rider with great control as your feel for the road is significantly enhanced In the same arena as the ride is the ease of use of the console which for me equals smoothness and ease The Thunderbirds radio and air console is incredibly well designed with everything within reach and intuitively organized Seem trivial to you Try changing the station at 75 miles an hour and see how important knob placement is 4 Trunk This is a important feature for me as I seem to move every 3 6 months The trunk on the Thunderbird is big enough for all of your luggage not to mention the corpse of Vinnie The Chin from a rival family My Defense I have read another review of this vehicle that criticizes the brake quality and I have to vehemently disagree with it I ride my brakes hard and I have never had a lockup or other incident The brakes do tend to squeek a bit but the noise is no indication of a performance issue The Final Verdict The bottom line is that the Thunderbird is a comfortable and well designed car at a reasonable price As long as you like burgundy vehicles and live in an area thats at least 30 Italian the Thunderbird is a great option
Pos I arrived in the states from Australia at the end of March 1999 to stay there for a year and come home at the end of March 2000 I stayed with friends in South Carolina who is a Ford man as I have always owned GM or Chevs they lent me a 1979 red corvette until I bought myself a car so after 3 months I did buy a 1985 Z28 Camaro 350 to fix up and use for the 9 months after looking at it I thought this was a bad idea so looked in the local paper and found a red 1991 V8 Thunderbird with 114 000 miles on it for 3000 After taking it for a test drive offered the lady 2800 and drove it home it had a slight water leak from the water pump so while replacing it I installed a set of under drive pulleys which I could notice the power increase the first time I drove it put a K amp N air cleaner in as well I had a friend come over from Australia so we drove from Greenville SC across to Sequin TX did about 3000 miles in that trip we took the long way and had no trouble at all and got 27 MPG sitting on 80 MPH had a radar detector it has a highway ratio in it 2 75 My brother came over from Australia so we went from Greenville SC down to Daytona Beach and back then drove across America to California which we did about 4600 miles trouble free When we left Williams AZ the car was buried under snow as we had a cold snap and snow dig the snow away and turn the key starting the engine at once and never missing a beat The car came with the premium sound system but the radio cassette was playing up so replaced it with a Pioneer radio CD I went to the wreckers and bought an electric motor seat assembly for the right hand side so when converting to RHD will have an electric adjustment Also bought a sports instrument cluster and centre handbrake assy out of a super coupe These cars were never made for export or for Right Hand Drive so have to get all the parts needed now for conversion I did 18 000 miles in 9 months without the car stopping or letting me down I gave the car 4 oil changes added fuel octane booster with every tank of gas it has the factory 15 alloy wheels with Michelin tyres I found the car very easy to drive and steer but did experience brake shudder which appears to be a common problem due to thin brake rotors I added a rear spoiler and had the windows tinted which makes the car look sporty as in Australia the only 2 door cars are mainly Jap imports so in the end I shipped the car back to Australia where I have to convert it over to Right Hand Drive for our road rules this cars owes me 10 000 Australia 5200 US landed back at my house in Australia which when converted to RHD they sell from 35 000 to 40 000 18 000 to 21 000 US BEST CAR I HAVE EVER OWNED
Pos This review is about Ford Mustang 3 8L Coupe with stick shift I test drove when I considered buying it I say considered because I did not buy it and here is why Test Drive The dealer talked too much during the test drive They always try to do that to distract you but I noticed the following things Styling You can argue but I think it could be better The car looks bulky the C pillars are thick which increases blind spots I was afraid to run over somebody while backing up the standard wheels look crude The previous Mustang looked more balanced Engine The 3 8L 193 hp engine does not seem all that powerful even with stick We went on the freeway onramp and I was disappointed Strange considering the 220 lb ft of torque rating at as low as 2800 rpm European and Japanese manufacturers manage to extract more than 200 hp out of 3 0 liter engines Note the A C was on during the test drive and was very efficient It might eat some power but not that much Transmission The shifter has quite short travel which is good but the clutch does not provide any feedback you cannot feel it engage by the pedal pressure or the dealer talked too much The clutch also engaged very high in the pedal travel I drove some Eastern European cars for several years and never had complaints like this one Or maybe im getting old and grumpy Suspension The suspension is not only stiff but creates a lot of unnecessary up and down motions The car uses live axle in the rear so I didnt expect much anyway Standard Equipment The list of standard equipment looks good It includes power windows mirrors locks and remote keyless entry alloy ugly wheels AM FM CD cassette player A C dual vanity mirrors etc Interior Interior materials fit and finish looks cheap I did not expect walnut for 16K but Ford could have done better As I said the C pillars are wide in coupe and the interior room is smaller than Id like The steering wheel tilts but does not telescope which might be a problem for the tall people Insurance and Safety Insurance rates are high especially if you are a male younger than 25 The crash test results are not encouraging either the overall rating is Acceptable with Poor death rate and Marginal injury rate Fuel Economy I didnt get a chance to see the actual fuel consumption myself but on paper its 19 MPG city 29 MPG highway Not impressive for the car of this size with manual transmission Warranty and Reliability Consumer Reports magazine says that Mustang has poor reliability Ford provides 36 000 mile 3 year warranty and 5 year corrosion warranty Majority of other manufacturers offers 60 000 mile 5 year powertrain warranty 100 000 mile 10 year warranty for Hyundai Kia The last three safety fuel economy and reliability also depend on the way you drive Pricing The price was good in theory I know that you can get the car for less than 16K at CarsDirect com for example but the particular dealership I went to wanted more than 17K and did not want to negotiate the price at all Besides they were very pushy and rude Needles to say they did not earn my business they didnt even try The dealer was constantly asking what monthly payment I can afford Well I can afford the payment I need to get better car I walked after which they called me several times asking how they can make me buy the car today I was unable to produce any kind of positive reply on this one I In car buying a lot depends on personal taste If you like Mustangs styling and features and decide to buy it it is a good deal providing you with electric everything remote keyless entry radio CD cassette V6 engine and alloy wheels for less than 16 If you want refinement fit and finish safety and reliability get ready to pay more for something else I
Code attempt:
from sklearn.feature_extraction.text import CountVectorizer
#instantiate the class
cv = CountVectorizer()
#list of sentences
for i in range(len(df['clean_Review'])):
text=df.loc[i, "clean_Review"]
#tokenize and build vocab
cv.fit(text)
print(cv.vocabulary_)
#transform the text
vector = cv.transform(text)
print(vector.toarray())
#df.loc[i,"porter"]=test
i=i+1

You don't need the looping. From the documentation:
from sklearn.feature_extraction.text import CountVectorizer
#instantiate the class
cv = CountVectorizer()
#vector is a sparse matrix storing individual words as "bag of words" model
vector = cv.fit_transform(df["clean_Review"].copy())
I assume that you have performed corpus cleaning step (lowercase, ascii encoding, stopword removal, etc) before using CountVectorizer to convert your model to bag of words, therefore I have kept the arguments of CountVectorizer() empty.
Example:
from sklearn.feature_extraction.text import CountVectorizer
import pandas as pd
#sample text corpus
corpus = pd.Series(["aa bb cc dd ee","bb cc dd ee","cc dd ee","dd ee","ee","ee ff"])
#instantiate the class
cv = CountVectorizer()
vector = cv.fit_transform(corpus)
print(corpus)
0 aa bb cc dd ee
1 bb cc dd ee
2 cc dd ee
3 dd ee
4 ee
5 ee ff
dtype: object
print(vector.toarray())
array([[1, 1, 1, 1, 1, 0],
[0, 1, 1, 1, 1, 0],
[0, 0, 1, 1, 1, 0],
[0, 0, 0, 1, 1, 0],
[0, 0, 0, 0, 1, 0],
[0, 0, 0, 0, 1, 1]])

How to skip items under labels while scraping website Python

I am trying to extract comments from a sample web forum. While extracting comments I would want to eliminate those which are under label "Quote". How can I ignore that and extract others as both are under same class
url = "https://www.f150forum.com/f118/would-you-buy-f150-again-463954/index3/"
page = requests.get(url, headers=headers)
soup = BeautifulSoup(page.text, 'html.parser')
domains = soup.find_all("div")
posts = soup.find(id = "posts")
comments_class = soup.findAll('div',attrs={"class":"ism-true"})
comments = [row.get_text() for row in comments_class]
Thanks in advance.

From that url this will filter comments that don't have Quote tag:
QUOTE = '\n\n\nQuote:\n\n\r\n\t\t\t'
without_quotes = [comment for comment in comments if QUOTE not in comment]

The problem is with your selector, you need to update it in order to avoid div which have label with text Quote in it. It will more be easily done using xpaths

When that element comes up, get the 3rd next div element:
import requests
from bs4 import BeautifulSoup
import re
url = "https://www.f150forum.com/f118/would-you-buy-f150-again-463954/index3/"
page = requests.get(url)
soup = BeautifulSoup(page.text, 'html.parser')
domains = soup.find_all("div")
posts = soup.find(id = "posts")
comments_class = soup.findAll('div',attrs={"class":"ism-true"})
comments_without_quote = []
for each in comments_class:
if 'Quote' in each.text:
nextComment = each.find('div', text=re.compile(r'Quote:')).findNext('div').findNext('div').findNext('div')
comments_without_quote.append(nextComment.text.strip())
comments_without_quote.append(each.text.strip())
Output:
print (comments_without_quote)
["I would say it depends. I really miss my 2014, I think that generation of truck was very well sorted out. beyond that I don't know. I am NOT buying a chevy or a ram. That leaves toyota. which holy crap that truck is ancient. I would just have to drive one. I really don't want to leave ford but this generation of truck is not proving to be the most reliable for me. I trade my trucks at 100k miles. I also don't believe that when toyota updates their truck next year that I want to be the guinea pig there either.", "Maybe the question should be...would you buy the same F150 again!\n\nI'm leaning towards an F250 with diesel.\n\nRealistically what are the options?\n\nFord\nGM/Chevy\nRam\nToyota\nNissan\n\nThats it! Unlike say SUVs where you may have 25 to choose from...full size trucks have five.", 'Sure would!', 'I am going to take a long hard look at Ram if they end up coming out with the rumored turbo\'d Inline 6 motor. I have had 4-5 Rams over the last few years as rentals and put a couple thousand miles on them combined and they are very appealing trucks. The 8 speed is fantastic, the ride is better than any stock F150 I have been in and, as you mentioned, they are better optioned for the price than an F150. Most of the ones I have had were BigHorns and they were all nicely optioned. I did have a 2019 Ram Rebel in San Antonio and that was a cool rig with the factory lift and 33" duratracs.', 'Quote:\n\n\r\n\t\t\tOriginally Posted by mass-hole\n\n\nI am going to take a long hard look at Ram if they end up coming out with the rumored turbo\'d Inline 6 motor. I have had 4-5 Rams over the last few years as rentals and put a couple thousand miles on them combined and they are very appealing trucks. The 8 speed is fantastic, the ride is better than any stock F150 I have been in and, as you mentioned, they are better optioned for the price than an F150. Most of the ones I have had were BigHorns and they were all nicely optioned. I did have a 2019 Ram Rebel in San Antonio and that was a cool rig with the factory lift and 33" duratracs.\n\nI\'ve been in a rental 2019 Big Horn the past week. It has heated steering wheel, power fold side mirrors, 4-auto (and it\'s a better 4-auto than my Lariat\'s by feel), remote start, heated seats, push button start (though no proxy entry), forward parking sensors, two tone soft touch interior, etc...and it\'s an XLT rival. That\'s kind of crazy.', 'Yup, as long as I need a half ton truck it will be an F-150.', "Quote:\n\n\r\n\t\t\tOriginally Posted by kozal01\n\n\nYup, as long as I need a half ton truck it will be an F-150.\n\nThis is also my response. My truck has been good to me, it's my first Ford and I like it compared to the last 3 trucks I've had since 2002 (Dodge, GMC, Titan). Although I'm thinking next year I may put the F250 option on the table.", 'Yes', "I've been in a rental 2019 Big Horn the past week. It has heated steering wheel, power fold side mirrors, 4-auto (and it's a better 4-auto than my Lariat's by feel), remote start, heated seats, push button start (though no proxy entry), forward parking sensors, two tone soft touch interior, etc...and it's an XLT rival. That's kind of crazy.", "Quote:\n\n\r\n\t\t\tOriginally Posted by blkZ28spt\n\n\nI've been in a rental 2019 Big Horn the past week. It has heated steering wheel, power fold side mirrors, 4-auto (and it's a better 4-auto than my Lariat's by feel), remote start, heated seats, push button start (though no proxy entry), forward parking sensors, two tone soft touch interior, etc...and it's an XLT rival. That's kind of crazy.\n\n\r\nHow can that be the XLT rival when you can't get some (most?) of those options on an XLT?\n\r\nMine will be a Ford.\n\r\nPeriod.", "I will probably buy Ford again. My truck is not perfect, but it has been pretty trouble free. It is certainly the quietest, most comfortable, most powerful vehicle I've owned.", "For me probably not. My decision was purely driven by the need to tow my trailer. I didn't want a 3/4 ton truck. I picked a half ton that could handle the most weight and power to pull, which this truck does well. However if I do decide to upgrade it'll be a Ram 2500, it's the best ride in a 3/4 ton. If not, no more trucks, I'm getting myself a Jaguar XJ.", 'I would. Been very happy with my 4x4 RCSB.']

Writing tuple output to a text file

Is it possible to write my output tuple to a text file? I am using following code to get text between two strings as write them to a text file:
def format_file(file, start, end):
f = open('C:\TEMP\Test.txt', 'r').read()
return tuple(x for x in ''.join(f.split(start)).replace('\n', '').split(end) if x != '')
print (format_file('XYZ', 'Q2 2016 Apple Inc Earnings Call - Final', 'Event Brief of Q1 2016 Apple Inc Earnings Call - Final'))
file = open('C:\TEMP\out.txt', 'w')
file.write(format_file('XYZ', 'Q2 2016 Apple Inc Earnings Call - Final', 'Event Brief of Q1 2016 Apple Inc Earnings Call - Final'))
But I keep getting following error:TypeError: write() argument must be str, not tuple.
When I try to return output as a string instead of a tuple I get a blank file. I would really appreciate any help on this one.
here is my input file text:
Q2 2016 Apple Inc Earnings Call - Final
OPERATOR: From Piper Jaffray, we'll hear from Gene Munster.
GENE MUNSTER, ANALYST, PIPER JAFFRAY & CO.: Good afternoon. Tim, can you talk a little bit about the iPhone ASP trends, and specifically you mentioned that the SE is going to impact, but how are you thinking about the aspirational market share that's out there, and your actual market share, and using price to close that gap? Is it just the SE or could there be other iPhone models that will be discounted, to try to be more aggressive in emerging markets?
And one for Luca. Can you talk a little bit about the services segment, in terms of what piece of the services is driving growth, and maybe a little bit about the profitability on a net basis versus the growth basis that you have referred to in the past. Thanks.
TIM COOK: I think the SE is attracting two types of customers. One is customers that wanted the latest technology, but wanted it in a more compact package. And we clearly see even more people than we thought in that category.
Secondly, it's attracting people aspire to own an iPhone, but couldn't quite stretch to the entry price of the iPhone, and we've established a new entry. I think both of these markets are very, very important to us, and we are really excited about where it can take us. I do think that we will be really happy with the new to iPhone customers that we see from here, because of the early returns we've had. We are currently supply constrained, but we'll be able to work our way out of this at some point. But it's great to see the overwhelming demand for it. I will let Luca comment on the ASPs.
LUCA MAESTRI: On the ASPs, Gene we mentioned that we were going to be down sequentially, and this is really the combination of two factors. So when we go from the March quarter to the June quarter, is the fact that we are having the SE entering the mix, and that obviously is going to have a downward pressure on ASP, and also this channel inventory reduction that we have talked about, obviously the channel inventory reduction will come from higher-end models, and that is also affecting the sequential trend on ASPs.
The question on services, when we look at our services business, obviously growing very well across the board. The biggest element, and the part of the services business that is growing very well, we mentioned 35%, is the App Store. It's interesting for us that our music business, which had been declining for a number of quarters, now that we have both a download model and a streaming model, we have now hit an inflection point, and we believe that this would be the bottom, and we can start growing from there over time.
We have many other services businesses that are doing very well, we have an iCloud business that is growing very quickly. Faster than the App Store, from a much lower base but I think it's important for us as we continue to develop these businesses. Tim have talked about Apple Pay. It doesn't provide a meaningful financial contribution at this point, but as we look at the amount of transactions that go into Apple Pay right now, and we think ahead for the long-term, that could be an interesting business for us, as well.
From a profitability standpoint, we have mentioned last time that when you look at it on a gross basis, so in terms of purchase value of these services, the profitability of the business is similar to Company average. Of course, when you met out the amount that is paid to developers, and you look at it, in terms of what is reported in our P&L, obviously that business has a profitability that is higher than Company average. We don't get into the specifics of specific products or services, but it is very clear it is significantly higher than Company average.
GENE MUNSTER: Thank you.
NANCY PAXTON: Thanks, Gene. Could we have the next question please?
OPERATOR: Katy Huberty with Morgan Stanley.
KATY HUBERTY, ANALYST, MORGAN STANLEY: Yes, thank you. First for Luca. This is the worst gross margin guide in a year and a half or so, and over the last couple of quarters, you have talked about number of tailwinds including component cost, the lower accounting deferrals that went into effect in September. You just mentioned the services margins are above corporate average. So the question is, are some of those tailwinds winding down? Or is a significant guide down in gross margin for the June quarter entirely related to volume and the 5 SE? And then I have a follow-up for Tim.
LUCA MAESTRI: Katy, clearly the commodity environment remains quite favorable, and we continue to expect cost improvements. The other dynamics that you have mentioned are still there, obviously what is different, and particularly as we look at it on a sequential basis coming out of the March quarter, we would have loss of leverage, and that obviously is going to have a negative impact on margins. The other factor that's important to keep in mind is this different mix of products.
Particularly when you look at iPhone, what I was mentioning to Gene earlier, I think we've got a couple of things that are affecting not only ASPs, but obviously, they also affects margins. And it's the fact that we have a channel inventory reduction at the top end of the range, and we've got the introduction of the iPhone SE at the entry level of the range. And so when you take into account those factors, those are the big elements that drive our guidance range right now.
KATY HUBERTY: Okay. Thank you. And that a question for Tim, appreciate the optimism around longer-term iPhone unit growth, but with developed market penetration in anywhere from 60% to 80%, the growth is going to have to come from new markets. You talked about India. Could you just spend a little bit more time on that market? What are some of the hurdles you have to overcome, for that to be a larger part of the business? When we expect Apple to have more distribution, and specifically your own stores in that country? Thanks.
TIM COOK: Katy, in the short term, let me just make a couple of comments on the developed markets, just to make sure this is clear. If you look at our installed base of iPhone today versus two years ago, it's increased by 80%. When you think about upgrade cycles, upgrade cycles would have varying rates on it. As I talked about on the comments, the iPhone 6s rate, upgrade rate is slightly higher than the iPhone 5s, but lower than the iPhone 6.
But the other multiplier in that equation is obviously the size of the installed base. The net of the idea is that I think there's still really, really good business in the developed markets, so I wouldn't want to write those off. It's our job to come up with great products that people desire, and also to continue to attract over Android switchers. With our worldwide share there's still quite a bit of room in the developed markets, as well.
From an India point of view, if you look at India, and each country has a different story a bit, but the things that have held not only us back, perhaps, but some others as well, is that the LTE rollout with India just really begins this year. So we will begin to see some really good networks coming on in India. That will unleash the power and capability of the iPhone, in a way that an older network, 2.5G or even some 3G networks, would not do. The infrastructure is one key one, and the second one is building the channel out.
Unlike the US as an example, where the carriers in the US sell the vast majority of phones that are sold in the United States, in India, the carriers in general sell virtually no phones, and it is out in retail, and retail is many, many different small shops. We've been in the process. It's not something we just started in the last few weeks.
We've been working in India now for a couple of years or more, but we've been working with great energy over the last 18 months or so, and I am encouraged by the results that we're beginning to see there, and believe there's a lot, lot more there. It is already the third largest smart phone market in the world, but because the smart phones that are working there are low-end, primarily because of the network and the economics, the market potential has not been as great there. I view India as where China was maybe 7 to 10 years ago from that point of view. I think there's a really great opportunity there.
NANCY PAXTON: Thank you, Katy. Could we have the next question please?
OPERATOR: We will go to Toni Sacconaghi with Bernstein.
TONI SACCONAGHI, ANALYST, BERNSTEIN: I have one, and then a follow-up, as well. My sense is that you talked about adjusting for the changes in channel inventory, that you are guiding for relatively normal sequential growth. And I think if you do the math it's probably the same or perhaps a touch worse in terms of iPhone unit growth sequentially, relative to normal seasonality between fiscal Q2 and Q3. I guess the question is, given that you should be entering new markets and you should see pronounced elasticity from the SE device, why wouldn't we be seeing something that was dramatically above normal seasonal, in terms of iPhone revenues and units for this quarter?
Maybe you could push back on me, but I can't help thinking that when Apple introduced the iPad Mini in a similar move, to move down market, there was great growth for one quarter, and the iPad never grew again and margins and ASPs went down. It looks like you are introducing the SE, and at least on a sequential basis, you not calling for any uplift, even adjusting for channel inventory, and ASPs I presume will go down and certainly it's impacting gross margins as you've guided to. Could you respond to, A, why you're not seeing the elasticity, and B, is the analogy with the iPad mini completely misplaced?
TIM COOK: Toni, it's Tim. Let me see if I can address your question. The channel inventory reduction that Luca referred to, the vast, vast majority of that is in iPhone. That would affect the unit compare that you maybe thinking about. The iPhone SE, we are thrilled with the response that we've seen on it.
It is clear that there is a demand there, even much beyond what we thought, and so that is really why we have the constraint that we have. Do I think it will be like the iPad Mini? No, I don't think so. I don't see that.
I think the tablet market in general, one of the challenges with the tablet market is that the replacement cycle is materially different than in the smart phone market. As you probably know, we haven't had an issue in customer satisfaction on the iPad. It is incredibly high, and we haven't had an issue with usage of the iPad. The usage is incredibly high.
But the consumer behavior there is you tend to hold on for very long period of time, before an upgrade. We continue to be very optimistic on the iPad business, and as I have said in my remarks, we believe we are going to have the best compare for iPad revenue this quarter that we have quite some time. We will report back in July on that one, but I think iPhone has a particularly different kind of cycle to it than the tablet market.
TONI SACCONAGHI: Okay, and if I could follow-up, Tim. You alluded to replacement cycles and differences between the iPad and the iPhone. My sense was, when you were going through the iPhone 6 cycle, was that you had commented that the upgrade cycle was not materially different. I think your characterization was that it accelerated a bit in the US, but international had grown to be a bigger part of your business, and replacement cycles there were typically a little bit longer. I'm wondering if it was only a modest difference between the 5s and the 6, how big a difference are we really seeing in terms of replacement cycles across the last three generations, and maybe you could help us, if the replacement cycle was flat this year relative to what you saw last year, how different would your results have been this quarter in the first half?
TIM COOK: There's a lot there. Let me just say I don't recall saying the things that you said I said about the upgrade cycle, so let me get that out of the way. Now let me describe without the specific numbers, the iPhone 6s upgrade cycle that we have measured for the first half of this year, so the first six months of our fiscal year to be precise, is slightly better than the rate that we saw with the iPhone 5s two years ago, but it's lower than the iPhone 6. I don't mean just a hair lower, it's a lot lower.
Without giving you exact numbers, if we would have the same rate on 6s that we did 6, there would -- it will be time for a huge party. It would be a huge difference. The great news from my point of view is, I think we are strategically positioned very well, because we have announced the SE, we are attracting customers that we previously didn't attract. That's really great, and this tough compare eventually isn't the benchmark. The install base is up 80% over the last two years, and so all of those I think bode well, and the switcher comments I made earlier, I wouldn't underestimate that, because that's very important for us in every geography. Thanks for the question.
NANCY PAXTON: Thanks, Toni. Can we have the next question please?
OPERATOR: From Cross Research Group, we'll hear from Shannon Cross.
SHANNON CROSS, ANALYST, CROSS RESEARCH: I have a couple of questions. One, Tim, can you talk a bit about what's going on in China? The greater China revenue I think was down 26%. You did talk about mainland China, but if you could talk about some of the trends you're seeing there, and how you think it's playing out, and maybe your thoughts on SE adoption within China as well.
TIM COOK: Shannon, thanks for the question. If you take greater China, we include Taiwan, Hong Kong, and mainland China in the greater China segment that you see reported on your data sheet. The vast majority of the weakness in the greater China region sits in Hong Kong, and our perspective on that is, it's a combination of the Hong Kong dollar being pegged to the US dollar, and therefore it carries the burden of the strength of the US dollar, and that has driven tourism, international shopping and trading down significantly compared to what it was in the year ago.
If you look at mainland China, which is one that I am personally very focused on, we are down 11% in mainland China, on a reported basis. On a constant currency basis, we are only down 7%, and the way that we really look at the health or underlying demand is look at sell-through, and if you look at there, we were down 5%. Keep in mind that is down 5% on comp a year ago that was up 81%.
As I back up from this and look at the larger picture, I think China is not weak, as has been talked about. I see China as -- may not have the wind at our backs that we once did, but it's a lot more stable than what I think is the common view of it. We remain really optimistic on China. We opened seven stores there during the quarter.
We are now at 35. We will open 5 more this quarter to achieve 40, which we had talked about before. And the LTE adoption continues to rise there, but it's got a long way ahead of it. And so we continue to be really optimistic about it, and just would ask folks to look underneath the numbers at the details in them before concluding anything. Thanks for the question.
SHANNON CROSS: Thanks. My second question is with regard to OpEx leverage, or thinking about when I look at the revenue, your revenue is below our expectations but OpEx is pretty much in line. So how are you thinking about potential for leverage, cost containment, maybe when macro is bad and revenue is under pressure, and how are you juggling that versus the required investment you need to go forward?
LUCA MAESTRI: It is Luca. Of course, we think about it. We think about it a lot, and so when you look at our results, for example, our OpEx for the quarter, for the March quarter was up 10%, which is the lowest rate that you have seen in years. And when you look within OpEx, you actually see two different dynamics. You see continued significant investments in research and development, because we really believe that's the future of the Company.
We continue to invest in initiatives and projects ahead of revenue. We had a much broader portfolio that we used to have. We do much more in-house technology development than we used to do a few years ago, which we think is a great investment for us to make. And so that parts we didn't need to protect, and we want to continue to invest in the business, right?
And then when you look at our SG&A portion of OpEx for the March quarter, it was actually down slightly. So obviously we think about it, and of course we look at our revenue trends, and we take measures accordingly. When you look at the guidance that we provided for the June quarter, that 10% year-over-year increase that I mentioned to you for the March quarter goes down to a range of 7% to 9% up, and again, the focus is on making investments in Road and continuing to run SG&A extremely tightly, and in a very disciplined way.
As you know, our E2R, expense to revenue ratio, is around 10%. It's something that we are very proud of, it's a number that is incredibly competitive in our industry, and we want to continue to keep it that way. At the same time, we don't want to under-invest in the business.
SHANNON CROSS: Thank you.
NANCY PAXTON: Thank you, Shannon. Could we have the next question please?
OPERATOR: From UBS we hear from Steve Milunovich.
STEVE MILUNOVICH, ANALYST, UBS: Tim, I first wanted to ask you about services and how do you view services? You've obviously highlighted it the last two quarters. Do you view it going forward as a primary driver of earnings, or do you view it, and you mentioned platforms in terms of your operating systems, which I would agree with. In that scenario I would argue it's more a supporter of the ecosystem, and a supporter of the hardware margins over time, and therefore somewhat subservient to hardware. It's great that it's growing, but longer-term, I would view its role as more creating an ecosystem that supports the high margins on the hardware, as opposed to independently driving earnings. How do you think about it?
TIM COOK: The most important thing for us, Steve, is that we want to have a great customer experience, so overwhelmingly, the thing that drives us are to embark on services that help that, and become a part of the ecosystem. The reality is that in doing so, we have developed a very large and profitable business in the services area, and so we felt last quarter and working up to that, that we should pull back the curtain so that people could -- our investors could see the services business, both in terms of the scale of it, and the growth of it. As we said earlier, the purchase value of the installed base services grew by 27% during the quarter, which was an acceleration over the previous quarter, and the value of it hit -- was just shy of $10 billion. It's huge, and we felt it was important to spell that out.
STEVE MILUNOVICH: Okay, and then going back to the upgrades of the installed base, you have clearly mentioned that you've pulled forward some demand, which makes sense, but there does seem to be a lengthening of the upgrade cycle, particularly in the US. AT&T and Verizon have talked about that. Investors I think perceive that maybe the marginal improvements on the phone might be less currently, and could be less going forward. At the same time, I think you just announced that you can get the upgrade program online, which I guess potentially could shorten it. Do you believe that upgrade cycles are currently lengthening, and can continue to do so?
TIM COOK: What we've seen is that it depends on what you compare it to. If you compare to the 5s, what we are seeing is the upgrade rate today is slightly higher, or that there are more people upgrading, if you will, in a similar time period, in terms of a rate, than the 5s. But if you compare to 6, you would clearly arrive at the opposite conclusion. I think it depends on people's reference points, and we thought it very important in this call to be very clear and transparent about what we're seeing. I think in retrospect, you could look at it and say, maybe the appropriate measure is more to the 5s, and I think everybody intuitively thought that the upgrades were accelerated with the 6, and in retrospect, when you look at the periods, they clearly were.
STEVE MILUNOVICH: Thank you.
NANCY PAXTON: Thanks, Steve. Could we have our next question, please?
OPERATOR: We will go to Rod Hall with JPMorgan.
ROD HALL, ANALYST, JPMORGAN: Yes, thanks for fitting me in. I wanted to start with a general, more general question. I guess, Tim, this one is aimed at you. As you think about where you thought things were going to head last quarter, when you reported to us, and how it's changed this quarter, obviously it's kind of a disappointing demand environment. Can you just help us understand what maybe the top two or three things are that have changed? And so as we walk away from this, we understand what the differences are, and what the direction of change is? Then I have a follow-up.
TIM COOK: I think you're probably indirectly asking about our trough comment, if you will, from last quarter. And when we made that, we did not contemplate or comprehend that we were going to make a $2 billion-plus reduction in channel inventory during this quarter. So if you factor that in and look at true customer demand, which is the way that we look at internally, I think you'll find a much more reasonable comparison.
ROD HALL: Okay, great. Thank you. And then for my follow-up, I wanted to ask you about the tax situation a little bit. Treasury obviously has made some rule changes, and I wonder, maybe if Luca, you could comment on what the impact to Apple from those is, if anything? and Tim, maybe more broadly how you see the tax situation for Apple looking forward? Thanks.
LUCA MAESTRI: Yes, Rod, these are new regulations, and we are in the processing of assessing them. Frankly from first read, we don't anticipate that they are going to have any material impact on our tax situation. Some of them relate to inversion transactions, obviously that's not an issue for us. Some of them are around internal debt financing, which is not something that we use, so we don't expect any issue there.
As you know, we are the largest US taxpayer by a wide margin, and we already pay full US tax on all the profits from the sales that we make in the United States, so we don't expect them to have any impact on us on tax reform. I will let Tim continue to provide more color, but we've been strong advocates for comprehensive corporate tax reform in this country. We continue to do that. We think a reform of the tax code would have significant benefits for the entire US economy, and we remain optimistic that we are going to get to a point where we can see that tax reform enacted. At that point in time, of course, we would have much more flexibility around optimizing our capital structure, and around providing more return of capital to our investors.
TIM COOK: The only thing I would add, Rod, is I think there are a growing number of people in both parties that would like to see comprehensive reform, and so I'm optimistic that it will occur. It's just a matter of when and that's difficult to say. But I think most people do recognize that it is in the US's interest to do this.
ROD HALL: Great, thanks.
NANCY PAXTON: Thank you, Rod. A replay of today's call will be available for two weeks as a podcast on the iTunes Store, as webcast on Apple.com/investor and via telephone. And the numbers for the telephone replay are 888-203-1112, or 719-457-0820, and please enter confirmation code 7495552. These replays will be available by approximately 5:00 PM Pacific time today.
Members of the press with additional questions can contact Kristin Huguet at 408-974-2414, and financial analysts can contact Joan Hoover or me with additional questions. Joan is at 408-974-4570, and I am at 408-974-5420. Thanks again for joining us.
OPERATOR: Ladies and gentlemen, that does conclude today's presentation. We do thank everyone for your participation.
[Thomson Financial reserves the right to make changes to documents, content, or other information on this web site without obligation to notify any person of such changes.
In the conference calls upon which Event Transcripts are based, companies may make projections or other forward-looking statements regarding a variety of items. Such forward-looking statements are based upon current expectations and involve risks and uncertainties. Actual results may differ materially from those stated in any forward-looking statement based on a number of important factors and risks, which are more specifically identified in the companies' most recent SEC filings. Although the companies may indicate and believe that the assumptions underlying the forward-looking statements are reasonable, any of the assumptions could prove inaccurate or incorrect and, therefore, there can be no assurance that the results contemplated in the forward-looking statements will be realized.
THE INFORMATION CONTAINED IN EVENT TRANSCRIPTS IS A TEXTUAL REPRESENTATION OF THE APPLICABLE COMPANY'S CONFERENCE CALL AND WHILE EFFORTS ARE MADE TO PROVIDE AN ACCURATE TRANSCRIPTION, THERE MAY BE MATERIAL ERRORS, OMISSIONS, OR INACCURACIES IN THE REPORTING OF THE SUBSTANCE OF THE CONFERENCE CALLS. IN NO WAY DOES THOMSON FINANCIAL OR THE APPLICABLE COMPANY OR THE APPLICABLE COMPANY ASSUME ANY RESPONSIBILITY FOR ANY INVESTMENT OR OTHER DECISIONS MADE BASED UPON THE INFORMATION PROVIDED ON THIS WEB SITE OR IN ANY EVENT TRANSCRIPT. USERS ARE ADVISED TO REVIEW THE APPLICABLE COMPANY'S CONFERENCE CALL ITSELF AND THE APPLICABLE COMPANY'S SEC FILINGS BEFORE MAKING ANY INVESTMENT OR OTHER DECISIONS.]
LOAD-DATE: April 29, 2016
LANGUAGE: ENGLISH
TRANSCRIPT: 042616a5987433.733
PUBLICATION-TYPE: Transcript
Copyright 2016 CQ-Roll Call, Inc.
All Rights Reserved
Copyright 2016 CCBN, Inc.
4 of 9 DOCUMENTS
FD (Fair Disclosure) Wire
January 26, 2016 Tuesday
Event Brief of Q1 2016 Apple Inc Earnings Call - Final
and the output I am expecting is everthing between 'Q2 2016 Apple Inc Earnings Call - Final' and 'Event Brief of Q1 2016 Apple Inc Earnings Call - Final' in a text file.

Did your try converting your tuple into a string and writing to the file?
s = str('XYZ', 'Q2 2016 Apple Inc Earnings Call - Final', 'Event Brief of Q1 2016 Apple Inc Earnings Call - Final')
file.write(s)

Reading data from CSV using pandas

I am trying to read data from a csv using pandas, like so :
import pandas as p
loadData = lambda f: np.genfromtxt(open(f,'r'), delimiter=',')
print "loading data.."
traindata = list(np.array(p.read_csv('FinalCSVFin.csv', delimiter=";"))[:,2])
I wish for this to give me a list of the 2nd column of the FinalCSVFin.csv. However, it is returning the error :
---------------------------------------------------------------------------
CParserError Traceback (most recent call last)
<ipython-input-7-de5ad26b44d2> in <module>()
7
8 print "loading data.."
CParserError: Error tokenizing data. C error: Expected 5 fields in line 3, saw 16
An extract of the CSV :
url;urlid;boilerplate;label;alexarank;;;;
http://www.bloomberg.com/news/2010-12-23/ibm-predicts-holographic-calls-air-breathing-batteries-by-2015.html;4042;"{""title"":""IBM Sees Holographic Calls Air Breathing Batteries ibm sees holographic calls, air-breathing batteries"",""body"":""A sign stands outside the International Business Machines Corp IBM Almaden Research Center campus in San Jose California Photographer Tony Avelar Bloomberg Buildings stand at the International Business Machines Corp IBM Almaden Research Center campus in the Santa Teresa Hills of San Jose California Photographer Tony Avelar Bloomberg By 2015 your mobile phone will project a 3 D image of anyone who calls and your laptop will be powered by kinetic energy At least that s what International Business Machines Corp sees in its crystal ball The predictions are part of an annual tradition for the Armonk New York based company which surveys its 3 000 researchers to find five ideas expected to take root in the next five years IBM the world s largest provider of computer services looks to Silicon Valley for input gleaning many ideas from its Almaden research center in San Jose California Holographic conversations projected from mobile phones lead this year s list The predictions also include air breathing batteries computer programs that can tell when and where traffic jams will take place environmental information generated by sensors in cars and phones and cities powered by the heat thrown off by computer servers These are all stretch goals and that s good said Paul Saffo managing director of foresight at the investment advisory firm Discern in San Francisco In an era when pessimism is the new black a little dose of technological optimism is not a bad thing For IBM it s not just idle speculation The company is one of the few big corporations investing in long range research projects and it counts on innovation to fuel growth Saffo said Not all of its predictions pan out though IBM was overly optimistic about the spread of speech technology for instance When the ideas do lead to products they can have broad implications for society as well as IBM s bottom line he said Research Spending They have continued to do research when all the other grand research organizations are gone said Saffo who is also a consulting associate professor at Stanford University IBM invested 5 8 billion in research and development last year 6 1 percent of revenue While that s down from about 10 percent in the early 1990s the company spends a bigger share on research than its computing rivals Hewlett Packard Co the top maker of personal computers spent 2 4 percent last year At Almaden scientists work on projects that don t always fit in with IBM s computer business The lab s research includes efforts to develop an electric car battery that runs 500 miles on one charge a filtration system for desalination and a program that shows changes in geographic data IBM rose 9 cents to 146 04 at 11 02 a m in New York Stock Exchange composite trading The stock had gained 11 percent this year before today Citizen Science The list is meant to give a window into the company s innovation engine said Josephine Cheng a vice president at IBM s Almaden lab All this demonstrates a real culture of innovation at IBM and willingness to devote itself to solving some of the world s biggest problems she said Many of the predictions are based on projects that IBM has in the works One of this year s ideas that sensors in cars wallets and personal devices will give scientists better data about the environment is an expansion of the company s citizen science initiative Earlier this year IBM teamed up with the California State Water Resources Control Board and the City of San Jose Environmental Services to help gather information about waterways Researchers from Almaden created an application that lets smartphone users snap photos of streams and creeks and report back on conditions The hope is that these casual observations will help local and state officials who don t have the resources to do the work themselves Traffic Predictors IBM also sees data helping shorten commutes in the next five years Computer programs will use algorithms and real time traffic information to predict which roads will have backups and how to avoid getting stuck Batteries may last 10 times longer in 2015 than today IBM says Rather than using the current lithium ion technology new models could rely on energy dense metals that only need to interact with the air to recharge Some electronic devices might ditch batteries altogether and use something similar to kinetic wristwatches which only need to be shaken to generate a charge The final prediction involves recycling the heat generated by computers and data centers Almost half of the power used by data centers is currently spent keeping the computers cool IBM scientists say it would be better to harness that heat to warm houses and offices In IBM s first list of predictions compiled at the end of 2006 researchers said instantaneous speech translation would become the norm That hasn t happened yet While some programs can quickly translate electronic documents and instant messages and other apps can perform limited speech translation there s nothing widely available that acts like the universal translator in Star Trek Second Life The company also predicted that online immersive environments such as Second Life would become more widespread While immersive video games are as popular as ever Second Life s growth has slowed Internet users are flocking instead to the more 2 D environments of Facebook Inc and Twitter Inc Meanwhile a 2007 prediction that mobile phones will act as a wallet ticket broker concierge bank and shopping assistant is coming true thanks to the explosion of smartphone applications Consumers can pay bills through their banking apps buy movie tickets and get instant feedback on potential purchases all with a few taps on their phones The nice thing about the list is that it provokes thought Saffo said If everything came true they wouldn t be doing their job To contact the reporter on this story Ryan Flinn in San Francisco at rflinn bloomberg net To contact the editor responsible for this story Tom Giles at tgiles5 bloomberg net by 2015, your mobile phone will project a 3-d image of anyone who calls and your laptop will be powered by kinetic energy. at least that\u2019s what international business machines corp. sees in its crystal ball."",""url"":""bloomberg news 2010 12 23 ibm predicts holographic calls air breathing batteries by 2015 html""}";0;345;;;;
http://www.popsci.com/technology/article/2012-07/electronic-futuristic-starting-gun-eliminates-advantages-races;8471;"{""title"":""The Fully Electronic Futuristic Starting Gun That Eliminates Advantages in Races the fully electronic, futuristic starting gun that eliminates advantages in races the fully electronic, futuristic starting gun that eliminates advantages in races"",""body"":""And that can be carried on a plane without the hassle too The Omega E Gun Starting Pistol Omega It s easy to take for granted just how insanely close some Olympic races are and how much the minutiae of it all can matter The perfect example is the traditional starting gun Seems easy You pull a trigger and the race starts Boom What people don t consider When a conventional gun goes off the sound travels to the ears of the closest runner a fraction of a second sooner than the others That s just enough to matter and why the latest starting pistol has traded in the mechanical boom for orchestrated electronic noise Omega has been the watch company tasked as the official timekeeper of the Olympic Games since 1932 At the 2010 Vancouver games they debuted their new starting gun which is a far cry from the iconic revolvers associated with early games it s clearly electronic but still more than a button that s pressed to get the show rolling About as far away as you can get probably while still clearly being a starting gun Pull the trigger once and off the Olympians go If it s pressed twice consecutively it signals a false start Working through a speaker system is what eliminates any kind of advantage for athletes It s not a big advantage being close to a gun but the sound of the bullet traveling one meter every three milliseconds could contribute to a win Powder pistols have been connected to a speaker system before but even then runners could react to the sound of the real pistol firing rather than wait for the speaker sounds to reach them This year s setup will have speakers placed equidistant from runners forcing the sound to reach each competitor at exactly the same time It wouldn t be an enormous difference Omega Timing board member Peter H\u00fcrzeler said in an email but when you think about reaction times being measured in tiny fractions of a second placing a speaker behind each lane has eliminated any sort of advantage for any athlete They all hear the start commands and signal at exactly the same moment There s also an ulterior reason for its look In a post September 11th world a gun on its way to a major event is going to raise more than a few TSA eyebrows even if it s a realistic looking fake Rather than deal with that the e gun can be transported while still maintaining the general look of a starting gun But there s still nothing like hearing a starting gun go off at the start of a race more than signaling the runners there s probably some Pavlovian response after more than a century of Olympic games that make people want to hear the real thing not a whiny electronic noise Everyone in the stands at home thankfully will still be getting that The sound is programmable and can be synthesized to sound like almost anything H\u00fcrzeler says but we program it to sound like a pistol it s a way to use the best possible starting technology but to keep a rich tradition alive and that can be carried on a plane without the hassle, too technology,gadgets,london 2012,london olympics,olympics,omega,starting guns,summer olympics,timing,popular science,popsci"",""url"":""popsci technology article 2012 07 electronic futuristic starting gun eliminates advantages races""}";1;5304;;;;
http://www.menshealth.com/health/flu-fighting-fruits?cm_mmc=Facebook-_-MensHealth-_-Content-Health-_-FightFluWithFruit;1164;"{""title"":""Fruits that Fight the Flu fruits that fight the flu | cold & flu | men's health"",""body"":""Apples The most popular source of antioxidants in our diet one apple has an antioxidant effect equivalent to 1 500 mg of vitamin C Apples are loaded with protective flavonoids which may prevent heart disease and cancer Next Papayas With 250 percent of the RDA of vitamin C a papaya can help kick a cold right out of your system The beta carotene and vitamins C and E in papayas reduce inflammation throughout the body lessening the effects of asthma Next Cranberries Cranberries have more antioxidants than other common fruits and veggies One serving has five times the amount in broccoli Cranberries are a natural probiotic enhancing good bacteria levels in the gut and protecting it from foodborne illnesses Next Grapefruit Loaded with vitamin C grapefruit also contains natural compounds called limonoids which can lower cholesterol The red varieties are a potent source of the cancer fighting substance lycopene Next Bananas One of the top food sources of vitamin B6 bananas help reduce fatigue depression stress and insomnia Bananas are high in magnesium which keeps bones strong and potassium which helps prevent heart disease and high blood pressure Next everything you need to know about cold and flu so you don\u2019t get sick this season, at men\u2019s health.com cold, flu, infection, sore throat, sneeze, immunity, germs, allergies, stay healthy, sick, contagious, medicines, cold medicine"",""url"":""menshealth health flu fighting fruits cm mmc Facebook Mens Health Content Health Fight Flu With Fruit""}";1;2663;;;;
What have I done incorrectly here ?

1) A better way
import pandas as pd
seperator = ";"
df = pd.read_csv("FinalCSVFin.csv", sep=seperator)
2) Your code
You define a function to Read a file using genfromtxt method in numpy, and then use pandas to read you file. I suggest the latter, just use read_csv method in pandas (as was described in 1).
3) Suggestions
Here are the points you can change to get your code working.
You implement a function to read data using np.genfromtxt. The problems are inconsistency in delimiter and also the lack of dtype in genfromtxt. I edit your function as follows:
loadData = lambda f, s: np.genfromtxt(open(f,'r'), dtype=None, delimiter=s)
This gives you a list of tuples. If your file (i.e. FinalCSVFin.csv) uses ";" as delimiter, call this function as follows:
valus = loadData("test.txt", ";")