owygs156

Online Resources for Neural Networks with Python

The neural network field enjoys now a resurgence of interest. New training techniques made training deep networks feasible. With deeper networks, more training data and powerful new hardware to make it all work, deep neural networks (or “deep learning” systems) suddenly began making rapid progress in areas such as speech recognition, image classification and language translation. [1]

As result of this there are many posts or websites over the web with the source code and tutorials for neural networks of different types and complexity. Starting from simple feedforward network with just one hidden layer the authors of blog posts or tutorials are helping us to understand how to build neural net (deep or shallow).

To help to find needed python source code for neural network with desired features the website Neural Networks with Python on the Web was created.

Please feel free to add any comments, suggestions or advise the link to neural network web page (python source code) via the comments box on this page.

References
1. Why artificial intelligence is enjoying a renaissance



Thinking Patterns and Computer Programs

This post is a continuation of previous post [1] where we started to look how computer programs can increase effective thinking. In this post we will look at some patterns of human thinking and how these patterns are implemented in the computer programs.

Humans often follow others in their actions. When we think about something we often are interesting how others thinking or doing for the same or similar subject. In computer science we can find different implementations of using this approach. For example recommender systems typically produce a list of recommendations in one of two ways – through collaborative and content-based filtering or the personality-based approach. Collaborative filtering approaches building a model from a user’s past behavior (items previously purchased or selected and/or numerical ratings given to those items) as well as similar decisions made by other users.[2] We can see something like this for example on Amazon website.

Thinking about situation from many different views is another useful technique. For example it could be useful think about not just what happens now but also what will happen next year or later. Or it might be useful to think about different group of users. In computer program we would need to add additional categories(attributes) to accomplish this.

According to Wikipedia there could be two types of thinking:convergent and divergent thinking. Convergent thinking involves aiming for a single, correct solution to a problem, whereas divergent thinking involves creative generation of multiple answers to a set problem. Divergent thinking is sometimes used as a synonym for creativity in psychology literature. Other researchers have occasionally used the terms flexible thinking or fluid intelligence. [3]

As a humans we might use systems thinking that can be viewed as a set of habits or practices within a framework that is based on the belief that the component parts of a system can best be understood in the context of relationships with each other and with other systems, rather than in isolation [4],[9],[10] Systems concepts are used widely in computer science – for example when we represent the system as black box, when we use feedback control or finite state machines.

Another technique for thinking is – structured thinking which is a process of putting a framework to an unstructured problem. Having a structure not only helps an analyst understand the problem at a macro level, it also helps by identifying areas which require deeper understanding. Structured Thinking allows us to map our ideas in structured fashion, thereby enabling us to identify which areas need the most attention. Mind mapping tools can help to implement structured thinking[5]. In computer science we can use decision tree to build structure from the data.

Dividing the problem in smaller problems is also useful technique that can be called as divide-and-conquer paradigm. This gives useful framework for thinking about problems. In mathematics or computer science it is used for solving problems recursively. [6]

Comparative-analysis – is the item-by-item comparison of two or more comparable alternatives, processes, products, qualifications, sets of data, systems, or the like. In accounting, for example, changes in a financial statement’s items over several accounting periods may be presented together to detect the emerging trends in the company’s operations and results. [7] In troubleshooting we often compare working device with not working device to identify the difference in the hope that it will help to understand why the device failed.

We also use organizing similar ideas or items into logical groupings. [8] By looking at differences or similarities between groups we can find new knowledge about items or group of items. It helps also to generalize our ideas or knowledge. In computer programming we can use clustering to group different items into groups. If we want to add new item to correct group we can use classification.

Thus we looked at different thinking patterns that are used by humans. Obviously computer programs has some limitations if we want to implement above patterns in our programs. However we saw that some patterns are implemented and are used in wide range of programming applications.

References
1. How Can We Use Computer Programming to Increase Effective Thinking
2. Recommender system
3. Creativity
4. Systems Thinking
5. 12 Free Mind Mapping Tools For a Data Scientist To Enhance Structured Thinking
6. Divide-and-Conquer Algorithms
7. Comparative-analysis
8. Affinity Diagram
9. General Systems Concepts
10. An Introduction to General Systems Thinking



How Can We Use Computer Programming to Increase Effective Thinking

Once a while we might find ourselves in situation when we think “I wish I knew this before” , “Why I did not think about this before” or “Why it took so long to come to this decision or action”. Can computer programs be used to help us to avoid or minimize the situations like this? Having background in computer science I decided to look at human thinking patterns and compare them with the learning computer algorithms.

The situations mentioned above as well as all our actions are result of our learning and thinking. Effective thinking and learning drive good decisions and actions.

As mentioned on Wikipedia [1] – “Learning is the act of acquiring new, or modifying and reinforcing, existing knowledge, behaviors, skills, values, or preferences and may involve synthesizing different types of information.”

Learning very closely connected to thinking. New information often can lead to new thoughts or ideas and during the thinking process we often come to the need to learn something new, to extend our knowledge.

“Thinking is a process of response to external stimuli, and if thinking is effective it results in changes to or strengthening of world views, beliefs, opinions, attitudes, behaviours, skills, understanding, and knowledge.
Thinking and learning have the same outcomes, so have to be very closely related.” [2]

Current computer algorithms can be very intelligent due to the latest advances in computer sciences. Computer programs can learn information and use this information for making intelligent decisions. There are a number of computer fields associated with learning. For example machine learning, deep learning, reinforcement learning successfully provide computer algorithms for learning information in many different applications.

After learning computers make decisions based on learned information and programming instructions created by programmers.
Computers can not think (at least as of right now). Human beings can think and they are very flexible in the process of making decisions. For example they can get new ideas or apply knowledge from totally different domain area.

While computers can not think, the computer programs can be very flexible – nothing stop us from combining several algorithms to cover all or most of all possibilities, nothing stop us to produce more intelligent program.
Just simple example – program can sort apple from pear based on color, or it can use color and shape. In the second case it will be more intelligent and more accurate. If needed we may be could add even more attributes like weight, smell.

Humans have the ability to think and foresee some future situations but not always use this ability. Often people make actions following same patterns or following other people or just picking the easy or random option. It can work well but not always. And here computers can help to humans – as the computer machines can access and process a lot of information and calculate different alternatives and choose optimal solution.

Computer programs use algorithms. Scientists create algorithm and then it coded into program. Can algorithm be created for increasing effective thinking? Different people use different ways of thinking , even for the same problem. However even for different problems, we can see common thinking patterns like following from simple to more complex, dividing the something complex into smaller pieces or using similarity. Some patterns are used often some not. Can we program those patterns? In the next post or posts we will take a look at learning and thinking patterns in the context of how they are programmed for the computers.

References
1. Wikipedia – Learning
2. The Relationship Between Thinking and Learning



Retrieving Emails from POP3 Server Using Python Script

My inbox has a lot of data as many websites are sending notifications and updates. So I tasked myself with creating python script to extract emails from POP3 email server and organize information in better way.
I started from the first step – automatically reading emails from mailbox. Based on examples I have found on Internet I created the script that is retrieving emails and removing not needed information like headers.
I am using web based email (not gmail). In the script I am using poplib module which encapsulates a connection to a POP3 server. Another module that I am using is email – this is a library for managing email messages. As I have many emails I limited for loop to 15 emails.

There are still a few things that can be done. For example I would like to keep “FROM:” data, also some HTML tags still need to be removed. However this code allows to extract body text from emails and can be used as starting point.

Feel free to provide any feedback or suggestions.

Here is the full source code for python script to get body text emails from mailbox.



import poplib
import email


SERVER = "server_name"   
USER = "email_address"
PASSWORD = "email_password"
 

server = poplib.POP3(SERVER)
server.user(USER)
server.pass_(PASSWORD)
 
 
numMessages = len(server.list()[1])
if (numMessages > 15):
    numMessages=15
for i in range(numMessages) :
    (server_msg, body, octets) = server.retr(i+1)
    for j in body:
        try:
            msg = email.message_from_string(j.decode("utf-8"))
            strtext=msg.get_payload()
            print (strtext)
        except:
            pass

References
1. Read Email, pop3
2. poplib — POP3 protocol client includes POP3 Example that opens a mailbox and retrieves and prints all messages
3. email — An email and MIME handling package



Getting WordNet Information and Building Graph with Python and NetworkX

WordNet and Wikipedia are often utilized in text mining algorithms for enriching short text representation [1] or for extracting additional knowledge about words. [2] WordNet’s structure makes it a useful tool for computational linguistics and natural language processing.[3] In this post we will look how to pull information from WordNet using python. Also we will look how to build graph for relations between words using python and NetworkX.

WordNet groups English words into sets of synonyms called synsets, provides short definitions and usage examples, and records a number of relations among these synonym sets or their members. WordNet can thus be seen as a combination of dictionary and thesaurus. While it is accessible to human users via a web browser, its primary use is in automatic text analysis and artificial intelligence applications. [4]

Here is how to get all synsets for the word ‘good’ using NLTK package:


from nltk.corpus import wordnet as wn

print (wn.synsets('good'))

#This is the output of above line:
#[Synset('good.n.01'), Synset('good.n.02'), Synset('good.n.03'), Synset('commodity.n.01'), Synset('good.a.01'), Synset('full.s.06'), Synset('good.a.03'), Synset('estimable.s.02'), Synset('beneficial.s.01'), Synset('good.s.06'), Synset('good.s.07'), Synset('adept.s.01'), Synset('good.s.09'), Synset('dear.s.02'), Synset('dependable.s.04'), Synset('good.s.12'), Synset('good.s.13'), Synset('effective.s.04'), Synset('good.s.15'), Synset('good.s.16'), Synset('good.s.17'), Synset('good.s.18'), Synset('good.s.19'), Synset('good.s.20'), Synset('good.s.21'), Synset('well.r.01'), Synset('thoroughly.r.02')]

All synsets are connected to other synsets by means of semantic relations. These relations, which are not all shared by all lexical categories, include:

hypernyms: Y is a hypernym of X if every X is a (kind of) Y (canine is a hypernym of dog)
hyponyms: Y is a hyponym of X if every Y is a (kind of) X (dog is a hyponym of canine)
meronym: Y is a meronym of X if Y is a part of X (window is a meronym of building)
holonym: Y is a holonym of X if X is a part of Y (building is a holonym of window) [4]

Here is how can we can get hypernyms and hyponyms from WordNet.

car = wn.synset(‘car.n.01’)
print (“HYPERNYMS”)
print (car.hypernyms())
print (“HYPONYMS”)
print (car.hyponyms())

Here is the output from above code:
HYPERNYMS
[Synset(‘motor_vehicle.n.01’)]
HYPONYMS
[Synset(‘ambulance.n.01’), Synset(‘beach_wagon.n.01’), Synset(‘bus.n.04’), Synset(‘cab.n.03’), Synset(‘compact.n.03’), Synset(‘convertible.n.01’), Synset(‘coupe.n.01’), Synset(‘cruiser.n.01’), Synset(‘electric.n.01’), Synset(‘gas_guzzler.n.01’), Synset(‘hardtop.n.01’), Synset(‘hatchback.n.01’), Synset(‘horseless_carriage.n.01’), Synset(‘hot_rod.n.01’), Synset(‘jeep.n.01’), Synset(‘limousine.n.01’), Synset(‘loaner.n.02’), Synset(‘minicar.n.01’), Synset(‘minivan.n.01’), Synset(‘model_t.n.01’), Synset(‘pace_car.n.01’), Synset(‘racer.n.02’), Synset(‘roadster.n.01’), Synset(‘sedan.n.01’), Synset(‘sport_utility.n.01’), Synset(‘sports_car.n.01’), Synset(‘stanley_steamer.n.01’), Synset(‘stock_car.n.01’), Synset(‘subcompact.n.01’), Synset(‘touring_car.n.01’), Synset(‘used-car.n.01’)]

Here is how to get synonyms, antonyms , lemmas and similarity: [5]


synonyms = []
antonyms = []

for syn in wn.synsets("good"):
    for l in syn.lemmas():
        synonyms.append(l.name())
        if l.antonyms():
            antonyms.append(l.antonyms()[0].name())

print(set(synonyms))
print(set(antonyms))
print (syn.lemmas())


w1 = wn.synset('ship.n.01')
w2 = wn.synset('cat.n.01')
print(w1.wup_similarity(w2))

Here is how we can use Textblob package [6] and represent some word relations via graph. The output graph is shown below.


from textblob import Word
word = Word("plant")
print (word.synsets[:5])
print (word.definitions[:5])

word = Word("computer")
for syn in word.synsets:
    for l in syn.lemma_names():
        synonyms.append(l)
        
import networkx as nx
import matplotlib.pyplot as plt
G=nx.Graph()


w=word.synsets[1]


G.add_node(w.name())
for h in w.hypernyms():
      print (h)
      G.add_node(h.name())
      G.add_edge(w.name(),h.name())


for h in w.hyponyms():
      print (h)
      G.add_node(h.name())
      G.add_edge(w.name(),h.name())

print (G.nodes(data=True))
plt.show()
nx.draw(G, width=2, with_labels=True)
plt.savefig("path.png")

Wordnet_graph

Here is the full source code


from nltk.corpus import wordnet as wn

print (wn.synsets('good'))

car = wn.synset('car.n.01')
print ("HYPERNYMS")
print (car.hypernyms())
print ("HYPONYMS")
print (car.hyponyms())

synonyms = []
antonyms = []

for syn in wn.synsets("good"):
    for l in syn.lemmas():
        synonyms.append(l.name())
        if l.antonyms():
            antonyms.append(l.antonyms()[0].name())

print(set(synonyms))
print(set(antonyms))
print (syn.lemmas())



w1 = wn.synset('ship.n.01')
w2 = wn.synset('cat.n.01')
print(w1.wup_similarity(w2))


from textblob import Word
word = Word("plant")
print (word.synsets[:5])
print (word.definitions[:5])

word = Word("computer")
for syn in word.synsets:
    for l in syn.lemma_names():
        synonyms.append(l)


import networkx as nx
import matplotlib.pyplot as plt
G=nx.Graph()


w=word.synsets[1]


G.add_node(w.name())
for h in w.hypernyms():
      print (h)
      G.add_node(h.name())
      G.add_edge(w.name(),h.name())
     



for h in w.hyponyms():
      print (h)
      G.add_node(h.name())
      G.add_edge(w.name(),h.name())



print (G.nodes(data=True))
plt.show()
nx.draw(G, width=2, with_labels=True)
plt.savefig("path.png")

References
1. Enriching short text representation in microblog for clustering
2. Automatic Topic Hierarchy Generation Using WordNet
3. WordNet
4. WordNet
5. WordNet NLTK Tutorial
6. Tutorial: What is WordNet? A Conceptual Introduction Using Python