.
Introduction
CurlWget Extension
Uploading data on Colab from any Website
Dealing with various file formats in Collaboratory
Save and reuse files without wasting the internet
If files are shared with you on gdrive
.
Read MoreThursday, August 05, 2021
Tuesday, April 06, 2021
Text Processing in Python Text using NLTK and spaCy
.
The Internet has connected the world, while Social Media like Facebook, Twitter and Reddit provided the platform for people to express their opinions and feelings toward a topic. Then, the proliferation of smartphones increased the usage of these platforms directly. For instance, there are 96% or 2,240 million Facebook active users who used Facebook by smartphones and tablets [1].
The increment in the usage of Social Media has grown the size of text data, and boost the studies or researches in Natural Language Processing (NLP), for example, Information Retrieval and Sentiment Analysis. Most of the time, the documents or the text files to be analyzed are gigantic and contains a lot of noise, directly used raw texts for analysis is inapplicable. Hence, text processing is essential to provide clean input for modelling and analysis.
Text processing contains two main phases, which are tokenization and normalization [2]. Tokenization is the process of splitting a longer string of text into smaller pieces, or tokens [3]. Normalization referring to convert number to their word equivalent, remove punctuation, convert all text to the same case, remove stopwords, remove noise, lemmatizing and stemming.
Stemming — removing affixes (suffixed, prefixes, infixes, circumfixes), For example, running to run
Lemmatization — capture canonical form based on a word’s lemma. For example, better to good [4]
.
Text Processing with NLTK
0. Import all needed libraries
1. Tokenization
2. Normalization
a. Removing Stop words
b. Lemmatization
3. Obtain the Cleaned Tokens
Text Processing with spaCy
1. Tokenization + Lemmatization
2. Normalization
a. Removing Noise
b. Removing Stopwords
3. Obtain the Cleaned Tokens
FULL:
https://towardsdatascience.com/text-processing-in-python-29e86ea4114c
References:
[1] M. Iqbal, “Facebook Revenue and Usage Statistics (2020),” 8 March 2021. [Online]. Available: https://www.businessofapps.com/data/facebook-statistics/.
[2] M. Mayo, “A General Approach to Preprocessing Text Data,” 2017. [Online]. Available: https://www.kdnuggets.com/2017/12/general-approach-preprocessing-text-data.html. [Accessed 12 June 2020].
[3] D. Subramanian, “Text Mining in Python: Steps and Examples,” 22 August 2019. [Online]. Available: https://medium.com/towards-artificial-intelligence/text-mining-in-python-steps-and-examples-78b3f8fd913b. [Accessed 12 June 2020].
[4] M. Mayo, “Natural Language Processing Key Terms, Explained,” 2017. [Online]. Available: https://www.kdnuggets.com/2017/02/natural-language-processing-key-terms-explained.html.
[5] “Natural Language Processing In Julia (Text Analysis),” JCharisTech, 1 May 2018. [Online]. Available: https://jcharistech.wordpress.com/2018/05/01/natural-language-processing-in-julia-text-analysis/.
[6] D. Jurafsky and J. H. Martin, “Speech and Language Processing,” 3 December 2020. [Online]. Available: https://web.stanford.edu/~jurafsky/slp3/.
[7] M.F. Goh, “Text Normalization with spaCy and NLTK,” 29 November 2020. [Online]. Available: https://towardsdatascience.com/text-normalization-with-spacy-and-nltk-1302ff430119.
Tuesday, January 05, 2021
Sentiment Analysis- Lexicon Models vs Machine Learning
.
Why are we even interested in Sentiment Analysis? Well, here’s situation to help you understand.
.
https://medium.com/nerd-for-tech/sentiment-analysis-lexicon-models-vs-machine-learning-b6e3af8fe746
Read MoreWednesday, August 12, 2020
What happened to PhoneGap? Why was it discontinued?
.
Adobe announced it is ending the development for PhoneGap and PhoneGap Build. PhoneGap was created in 2008 to give mobile app developers a way to easily create web and mobile applications with a single codebase.
Since 2008, the industry and market has evolved and PhoneGap usage has declined.
“In the context of these developments and declining PhoneGap usage, Adobe is focusing on providing a platform that enables developers to build, extend, customize and integrate with Adobe products,” the company wrote in a post.
PhoneGap Build will be discontinued on October 1. In addition, Adobe will be ending its investment in Apache Cordova.
According to the company, Apache Cordova will continue to exist, and users can also transition to Ionic, or alternative products such as Monaca, Framework7, NativeScript or Progressive Web apps. Additionally, PhoneGap will continue to be free and open source, but it won’t be actively worked on. Eventually, Adobe explained that PhoneGap tooling will become incompatible with the latest version of Apache Cordova.
“As the pioneer of hybrid app development, aka web developers building mobile apps, this is truly the end of an era,” Max Lynch, co-founder and CEO of Ionic, wrote in a post. “But one thing that hasn’t changed: web developers want to build web apps and run them everywhere. They want to use their existing skills, browser-based development process, web libraries, and code to build mobile apps.”
.
https://sdtimes.com/adobe/adobe-to-discontinue-phonegap-development/
Monday, August 10, 2020
What happened to Nokia? Why was it discontinued?
.
Way back before phones had apps, touchscreens, or cameras, one Finnish brand led the mobile phone revolution.
Renowned for its indestructible build and multiday battery, Nokia swiftly captured market share by promoting mobile phones as fashion accessories. Just like with watch straps, users could swap cover colors, turning a cellphone into a fashion statement.
By 1998, Nokia overtook Motorola to become the world’s largest mobile phone brand.
At its pinnacle in 2007, Nokia had 51% of global market share in mobile phones. To put that into context, Apple now has roughly 25% of global market share.
From the highs of global dominance to the lows of nearing bankruptcy, Nokia’s phone business culminated in a sale to Microsoft for $7.2 billion in 2013.
“We didn’t do anything wrong, but somehow, we lost” — Stephen Elop, former Nokia CEO
Nokia’s Mistakes
The brand’s downfall stems from numerous factors — both controllable and uncontrollable. Either way, they certainly messed something up.
Here are four key mistakes that lead to the company’s decline:
1. Only Focusing on Hardware
Building a physical device such as a mobile phone is undoubtedly a feat. However, without good software in it, it wasn’t going to stick.
By the time they realized it, Android and Apple already had the first-mover advantage into the app-based software (iOS and Android) that is now widespread in the phone industry.
Nokia’s operating system was called Symbian. While iOS and Android were app-based, Symbian was device-based. Basically, they were headed in the wrong direction. By 2009, Nokia was using 57 different and incompatible versions of its operating system — a complete nightmare for the company.
It must be for a reason there’s a huge buzz about software engineers, yet you hardly hear about “hardware engineers.”
2. Reluctance to Change Operating Systems
Image of Microsoft CEO Steve Ballmer and Nokia CEO Stephen Elop
Microsoft CEO Steve Ballmer and Nokia CEO Stephen Elop. Image by BI.
As Google entered the market in 2008, many competitors jumped ship to the Android operating system. Among them were soon-to-be bestsellers Samsung, Motorola, and Huawei. While competitors enjoyed an increasing share of the market, Nokia was reluctant to switch operating systems.
In 2011, Nokia finally took the leap. Except they didn’t change for Android. They partnered with Microsoft to implement Windows Phone as Nokia’s primary operating system.
This move proved to be catastrophic for the company, with Nokia effectively surrendering its market-leading position in 2013. To put this mistake into context, Android now dominates the OS market with over 80% of the smartphone operating systems, while Windows Phone is now discontinued.
Finally, in 2014, Nokia came to its senses and made the switch to Android, but it was all too late.
3. Dysfunctional Organization
Image of an Engineering Matrix Structure
Engineering Matrix Structure. Chart by PMI
While Nokia operated using a standard hierarchical organization through to the early 2000s, company executives decided to reorganize into a matrix structure in 2004, with the hopes of increasing innovation.
With the reorganization, conflicts began to arise. Among the disadvantages of having a matrix structure is that several managers have equal authority, which inevitably leads to power struggles. Inevitably, this also led to the departures of key members of the executive team.
While Nokia was dwelling with internal conflict, Apple — its biggest threat — was managed by a laser-sharp Steve Jobs, and was swiftly gaining market share through meticulous design, branding, and execution.
4. Missing the Smartphone Wave
Image of the first iPhone in 2007
The first iPhone back in 2007. Note the 4–16GB storage options; how times have changed. Image by iPhoneLife
In the mid-2000s, when smartphones began to emerge, Nokia sat comfortably on the cellphone throne. They were established, popular, and profitable.
Blinded by their success, they missed out on the first-mover advantage for smartphones. Given their resources, Nokia could have gained a serious lead over competitors in this area. Instead, they focused on producing durable, affordable, and traditional cellphones; the exact opposite of what most consumers were looking for.
Catapulted by the sleek iPhone, smartphones swiftly took over, kicking Nokia out of the mobile phone podium.
Read MoreMonday, March 23, 2020
Get Started with Google Colab for Machine Learning and Deep Learning
.
What is Google Colab?
GPUs and TPUs on Colab
Getting Started with Google Colab
Google Colab Runtimes
Using Terminal Commands on Google Colab
Cloning Repositories in Google Colab
Uploading Files and Datasets
Saving Your Notebook
Exporting Data/Files from Google Colab
Sharing Your Notebook
.
https://www.analyticsvidhya.com/blog/2020/03/google-colab-machine-learning-deep-learning/
Read MoreWednesday, January 01, 2020
What happened to Floppy Diskette? Why was it discontinued?
.
A floppy disk is a disk storage medium composed of a disk of thin and flexible magnetic storage medium encased in a rectangular plastic carrier. It is read and written using a floppy disk drive (FDD). Floppy disks were an almost universal data format from the 1970s into the 1990s, used for primary data storage as well as for backup and data transfers between computers.
.
Floppy disks remained a popular medium for nearly 40 years, but their use was declining by the mid- to late 1990s. The introduction of high speed computer networking and formats based on the new NAND flash technique (like USB flash drives and memory cards) led to the eventual disappearance of the floppy disk as a standard feature of microcomputers, with a notable point in this conversion being the introduction of the floppy-less iMac in 1998. After 2000, floppy disks were increasingly rare and used primarily with older hardware and especially with legacy industrial computer equipment.
.
https://en.wikipedia.org/wiki/History_of_the_floppy_disk
Tuesday, December 31, 2019
Monday, May 06, 2019
Microsoft Visual Studio Online - an online code editor
.
Use the full power of Visual Studio Code, including the editor, terminal, debugger, GitHub Copilot, version control, settings sync, and the entire ecosystem of extensions. Work in the browser or hand off to your desktop.
.
Read MoreSunday, November 25, 2018
API for Google Ngram Viewer
.
What is the API for Google Ngram Viewer?
The Google Ngram Viewer shows the frequency of phrases over time. It has an API, but it’s not documented. In the Google Ngram Viewer site, if you search for the frequency of “Churchill” between 1800 and 2000, it will take you to a page at this URL:
https://books.google.com/ngrams/graph?content=Churchill&year_start=1800&year_end=2000&corpus=26&smoothing=3
This URL gives you an HTML page showing a chart. To turn this into an API, just replace the graph in the URL with json. For example, here’s the same query from the CLI, showing the results as JSON:
$ curl -s 'https://books.google.com/ngrams/json?content=Churchill&year_start=1800&year_end=2000&corpus=26&smoothing=3' | jq . | head
[
{
"ngram": "Churchill",
"parent": "",
"type": "NGRAM",
"timeseries": [
3.4028656727969064e-06,
3.4380268971290205e-06,
3.466722167407473e-06,
3.4695450982066437e-06,
Thanks to Frans Badenhorst for this solution! What follows is my original solution, which is less elegant.
const fetch = require('node-fetch');
const regexp = /var data = (.*);/;
async function fetchNgram(phrases) {
console.log("ngram(", phrases, ")");
const params = new URLSearchParams();
params.set('content', phrases.join(','));
params.set('year_start', '1800');
params.set('year_end', '2000');
params.set('corpus', '15'); // English
params.set('smoothing', '0');
const response = await fetch('https://books.google.com/ngrams/graph?' + params.toString());
const responseText = await response.text();
const match = regexp.exec(responseText);
const json = match[1];
return JSON.parse(json);
}
module.exports = { fetchNgram };
Here’s an example of usage, showing the frequency of “Churchill” between 1800 and 2000:
$ node
> const ngram = require('./ngram.js');
undefined
> ngram.fetchNgram(['churchill']).then(results => console.log(results))
...
[ { ngram: 'churchill',
type: 'NGRAM',
timeseries:
[ 0,
0,
0,
...
0,
5.409022563185317e-9,
0,
... 101 more items ],
parent: '' } ].
Read More
Subscribe to:
Comments (Atom)