Some Funny Linux Command Line Concoctions !
One cannot say it enough that the terminal is a very powerful tool and probably the most interesting part of linux.
1. Aptitude Cowsay ?!
You can also try "cowthink"
2. Fortune
Fortune displays random sentence.
1. Aptitude Cowsay ?!
$sudo apt-get install cowsayNow run something like:
$cowsay Hi there!
You can also try "cowthink"
2. Fortune
Fortune displays random sentence.
$sudo apt-get install fortune-modNow simply run "fortune"
Now try combination like:
$fortune -l | cowthink
To change the image to something other than cow, type the following to know all possibilities:
$cowsay -l
Now choose any character you like and run:
$fortune -s | cowthink -f dragon
3. Insult with "sudo" !?
To do so, first edit the sudoer file with:
$sudo visudoThe sudoer file opens in the terminal. Near the top, add a line:
Defaults insultsNow save and close the file. Clear the cache and then mistype your password:
$sudo -k $sudo ls
Dynamic Site Development With Django Framework (Part - 1)
Let me start by defining: What is a "Web Application Framework" ?
A web application framework (WAF) is a software framework that is designed to support the development of dynamic websites, web applications, web services and web resources. Framework aims to alleviate the overhead associated with common activities performed in web development.
Django is a free and open source web application framework, written in Python. When building a website, you always need a similar set of components for example: a way to handle user authentication (signing up, signing in, signing out), a management panel for your website, forms, a way to upload/download files etc. Fortunately for you other people long ago noticed that web developers face similar problems when building a new site, so they teamed up and created frameworks. Django gives you ready-made components you can use. Frameworks exist to save you from having to reinvent the wheel and help alleviate some of the overhead when you’re building a new site.
Django Installation
Before installing Django, we need to set up our development environment. You may skip this step but its highly recommended not to. So lets create virtual environment (also called virtualenv). It will sandbox (or isolate) your Python/Django setup on a project wise basis, i.e that any changes you make to one website won't affect any others that you are also developing. The installation procedure described below is for Linux /OS X.
In case of error on Linux (Ubuntu or derivatives) try:
A myenv (or choose any name you like) directory will be create. To start or activate it, run:
mkdir django_project cd django_project python3 -m venv myenv
In case of error on Linux (Ubuntu or derivatives) try:
sudo apt-get install python-virtualenv virtualenv --python=python3.4 myenv
A myenv (or choose any name you like) directory will be create. To start or activate it, run:
source myenv/bin/activate
Note 'source' might not be available sometimes. So try this instead:
. myenv/bin/activate
Now virtual environment is all set up. Time to install Django.
pip install django==1.8
You can specify particular version of Django to installed (here 1.8).
Create Django Project
Now to create django project, run:
The django-admin.py script will create directories for you. The tree structure of directories should look like this:
django_project
|--manage.py
|--mysite
settings.py
urls.py
wsgi.py
__init__.py
Create Django Project
Now to create django project, run:
django-admin startproject mysite .
The django-admin.py script will create directories for you. The tree structure of directories should look like this:
django_project
|--manage.py
|--mysite
settings.py
urls.py
wsgi.py
__init__.py
E-mail Automation using Python (Part - 2)
In this post I'll will be covering retrieving email using Python script:
Just as SMTP is the protocol for sending email, the Internet Message Access Protocol (IMAP) specifies how to communicate with an email provider’s server to retrieve emails sent to your email address. Python comes with an imaplib module, but in fact the third-party imapclient module is easier to use. This is an introduction to use IMAPClient. The full documentation is at imapclient.readthedocs.org.
The imapclient module downloads emails from an IMAP server in a rather complicated format. Most likely, we’ll want to convert them from this format into simple string values. The pyzmail module does the hard job of parsing these email messages for us. Complete documentation for PyzMail at www.magiksys.net/pyzmail.
On line 4, we import all necessary libraries. imapclient for creating IMAP object, pymail for parsing mails, getpass to get password in hidden mode, pprint(stands for pretty print) to display formatted output.
Then we establish connection to gmail's imap client by creating a imapobj on line 6 and enable SSL encryption.
In lines 10-12 mail id and password of user is accepted and authenticated. On line 15, we print all the folders present in the user's gmail (like: INBOX, sent, trash etc.). The user can choose which folder he/she wants to explore or retrieve mail from.
On line 20th, sender's mail id accepted from user. Here the user has choice that he/she can either enter mail id of any one contact or 'ALL' to retrieve mail from all.
On line 21st the program asks the user to enter date from which he/she wants to retrieve mail. The format of date to be followed must be 'dd-MMM-YYYY' like '01-JAN-2015'.
The search() doesn't return mail themselves but rather unique IDs (UIDs)for emails, as integer values. We can then pass the UIDs to fetch() to obtain email content.
SIZE LIMIT
If your search matches a large number of emails, Python might raise an exception that says: imapib.error: got more than 10000 bytes. When this happens, we will have to disconnect and re-connect IMAP server and try again.
This limit is in place to prevent Python programs from eating too much memory. We can change this limit to (say)10,000,000 bytes by running this code:
Now in lines 30-35 we pass each UID and parse the mail components (like body, subject etc) and print it in a neat format in lines 36-40.
When our program has finished retrieving mails, we simply call IMAPClient's logout() to disconnect from IMAP server.
If your program runs for several minutes or more, IMAP server may time out or automatically disconnect. In this case the method call our program make will throw exception like:- imaplib.abort: socket error: [Winerror] An existing connection was forcibly closed by the remote host.
Your program will have to call imapclient.IMAPClient() to re-connect.
Now you might be thinking where is the automation part in all of this...well the automation part will be exactly same as that of Email Automation (Part-1).
You can choose when to retrieve mail and then take action accordingly. You can even write a script in which based on (say) subject of a particular mail, you can automate it to do perform another action(for example: you can remotely command your bot to perform certain task!)
The imapclient module downloads emails from an IMAP server in a rather complicated format. Most likely, we’ll want to convert them from this format into simple string values. The pyzmail module does the hard job of parsing these email messages for us. Complete documentation for PyzMail at www.magiksys.net/pyzmail.
#!/usr/bin/python
# coding: utf-8
def retmail():
import imapclient, time, pprint, pyzmail, getpass
print "Establishing connection to gmail..."
imapobj = imapclient.IMAPClient('imap.gmail.com', ssl=True)
print "Encryption enabled to gmail."
time.sleep(1)
mailid = raw_input("\nEnter your mail id to login: ")
pswd = getpass.getpass("Enter your password: ")
imapobj.login(mailid, pswd)
print "\nList of folders: (loading...)\n"
for i in range(len(imapobj.list_folders())):
print imapobj.list_folders()[i][2] + ',',
folder = raw_input("\nEnter folder you want(eg: INBOX): ")
imapobj.select_folder(folder.upper(), readonly=True)
sendername = raw_input("Enter sender's mail id (eg:choose 'ALL' to retrieve all mails):")
since_date = raw_input("Enter date since which you want to receive the mail(eg: 01-JAN-2015): ")
if sendername == 'ALL':
UIDs = imapobj.search(['SINCE ' + since_date, sendername ])
else:
UIDs = imapobj.search(['SINCE ' + since_date, 'FROM ' + sendername ])
l = len(UIDs)
rawmsg = imapobj.fetch(UIDs, ['BODY[]'])
for i in range(l):
msg = pyzmail.PyzMessage.factory(rawmsg[UIDs[i]]['BODY[]'])
subject = msg.get_subject()
fromaddr = msg.get_address('from')
body = msg.text_part.get_payload().decode(msg.text_part.charset)
print '\nMessage ' + str(i+1) + ': '
print 'Subject: ' + subject
print 'From: ' + fromaddr[0] + ' <' + fromaddr[1] + '>'
print 'Body:\n'+ body
imapobj.logout()
print "Logout requested..."
time.sleep(1)
print "Logout successful"
retmail()
On line 4, we import all necessary libraries. imapclient for creating IMAP object, pymail for parsing mails, getpass to get password in hidden mode, pprint(stands for pretty print) to display formatted output.
Then we establish connection to gmail's imap client by creating a imapobj on line 6 and enable SSL encryption.
In lines 10-12 mail id and password of user is accepted and authenticated. On line 15, we print all the folders present in the user's gmail (like: INBOX, sent, trash etc.). The user can choose which folder he/she wants to explore or retrieve mail from.
On line 20th, sender's mail id accepted from user. Here the user has choice that he/she can either enter mail id of any one contact or 'ALL' to retrieve mail from all.
On line 21st the program asks the user to enter date from which he/she wants to retrieve mail. The format of date to be followed must be 'dd-MMM-YYYY' like '01-JAN-2015'.
The search() doesn't return mail themselves but rather unique IDs (UIDs)for emails, as integer values. We can then pass the UIDs to fetch() to obtain email content.
SIZE LIMIT
If your search matches a large number of emails, Python might raise an exception that says: imapib.error: got more than 10000 bytes. When this happens, we will have to disconnect and re-connect IMAP server and try again.
This limit is in place to prevent Python programs from eating too much memory. We can change this limit to (say)10,000,000 bytes by running this code:
import imaplib imaplib.MAXLINE = 10000000
Now in lines 30-35 we pass each UID and parse the mail components (like body, subject etc) and print it in a neat format in lines 36-40.
When our program has finished retrieving mails, we simply call IMAPClient's logout() to disconnect from IMAP server.
If your program runs for several minutes or more, IMAP server may time out or automatically disconnect. In this case the method call our program make will throw exception like:- imaplib.abort: socket error: [Winerror] An existing connection was forcibly closed by the remote host.
Your program will have to call imapclient.IMAPClient() to re-connect.
Now you might be thinking where is the automation part in all of this...well the automation part will be exactly same as that of Email Automation (Part-1).
You can choose when to retrieve mail and then take action accordingly. You can even write a script in which based on (say) subject of a particular mail, you can automate it to do perform another action(for example: you can remotely command your bot to perform certain task!)
E-mail Automation using Python (Part - 1)
In this post I'll be covering sending and scheduling email sending using Python script:
Checking and replying to emails is a huge time sink. We cannot simply write a program to handle all our email cause obviously each message requires its own response.
Checking and replying to emails is a huge time sink. We cannot simply write a program to handle all our email cause obviously each message requires its own response.
For example you have a list of people that you want to send say a standard birthday invitation to (with of course different name on each mail) depending on various parameters like their age, location etc. Typing and sending to each one of them would take a lot of time and patience. Fortunately we can automate this task by writing a simple python script.
Much like HTTP is the protocol used by computers to send web pages across the internet, Simple Mail Transfer Protocol (SMTP) is the protocol used for sending email. SMTP dictates how email messages should be formatted, encrypted, and relayed between mail servers, and all the other details that your computer handles after you click send. All these technical details are simplified into a few functions by Python’s smtplib module.
# coding: utf-8
import time,datetime
print "\nEnter time you want to send mail:\n\t1. Now\n\t2. Morning (8:00 AM)\n\t3. Afternoon (1:00 PM)\n\t4. Evening (6:00 PM)\n\t5. Night (9:00 PM)\n\t6. Customise\n"
choice = raw_input(":")
present_time = datetime.datetime.now()
send_time = datetime.datetime.now()
def now():
global present_time
print "Time now is: ",
print present_time.strftime('%I:%M:%S %p')
print "Date: ",
print present_time.strftime('%A %d %b %Y')
send_time = present_time
def morning():
global send_time, present_time
print "\nTime now is: ",
print present_time.strftime('%I:%M:%S %p')
print "Date: ",
print present_time.strftime('%A %d %b %Y')
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day, 8, 0,0)
if present_time > send_time:
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day + 1, 8, 0,0)
print "Mail will be sent: ",
print send_time.strftime('%I:%M:%S %p') + ' on',
print send_time.strftime(' %A %d %b %Y')
def afternoon():
global send_time, present_time
print "\nTime now is: ",
print present_time.strftime('%I:%M:%S %p')
print "Date: ",
print present_time.strftime('%A %d %b %Y')
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day, 13, 0,0)
if present_time > send_time:
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day + 1, 13, 0,0)
print "Mail will be sent: ",
print send_time.strftime('%I:%M:%S %p') + ' on',
print send_time.strftime(' %A %d %b %Y')
def evening():
global send_time, present_time
print "\nTime now is: ",
print present_time.strftime('%I:%M:%S %p')
print "Date: ",
print present_time.strftime('%A %d %b %Y')
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day, 18, 0,0)
if present_time > send_time:
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day + 1, 18, 0,0)
print "Mail will be sent: ",
print send_time.strftime('%I:%M:%S %p') + ' on',
print send_time.strftime(' %A %d %b %Y')
def night():
global send_time, present_time
print "\nTime now is: ",
print present_time.strftime('%I:%M:%S %p')
print "Date: ",
print present_time.strftime('%A %d %b %Y')
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day, 21, 0,0)
if present_time > send_time:
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day + 1, 21, 0,0)
print "Time at which mail will be sent: ",
print send_time.strftime('%I:%M:%S %p') + ' on',
print send_time.strftime(' %A %d %b %Y')
def custom():
global send_time, present_time
print "\nTime now is: ",
print present_time.strftime('%I:%M:%S %p')
print "Date: ",
print present_time.strftime('%A %d %b %Y')
print "\nCustomise Time(24hr format): "
hr = int(raw_input("Enter hour: "))
mn = int(raw_input("Enter minutes: "))
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day, hr, mn,0)
if present_time > send_time:
send_time = datetime.datetime(datetime.date.today().year, datetime.date.today().month, datetime.date.today().day + 1, hr, mn,0)
print "\nMail will be sent at: ",
print send_time.strftime('%I:%M:%S %p') + ' on',
print send_time.strftime(' %A %d %b %Y')
options = {
'1':now,
'2':morning,
'3':afternoon,
'4':evening,
'5':night,
'6':custom,
}
options[choice]()
print "\nEnter mail details: ...(loading)"
import smtplib, time, getpass
smtpobj = smtplib.SMTP('smtp.gmail.com', 587)
if type(smtpobj.ehlo()[0]) == int:
print "\nConnection established to Gmail"
#else:
# break
if type(smtpobj.starttls()[0]) == int:
print "Encryption enabled"
#else:
# break
time.sleep(1)
mailid = raw_input("\nEnter your mail id: ")
pswd = getpass.getpass("Enter your password: ")
if smtpobj.login(mailid, pswd)[0] == int:
print id + ' authenticated successfully.'
receiver_addr = raw_input("\nEnter receiver's address: ")
sub_of_mail = raw_input("Enter subject: ")
body_of_mail = raw_input("Enter body: ")
print "\nMail will be sent on ",
print send_time.strftime('%I:%M:%S %p') + ' on',
print send_time.strftime(' %A %d %b %Y')
#........Time to sleep..........#
for i in range(int(round((send_time - present_time).total_seconds()))):
time.sleep(1)
#........Sleep over.............#
smtpobj.sendmail(mailid, receiver_addr, 'Subject:' + sub_of_mail + '\n' + body_of_mail)
print "Sending"
for i in range(5):
print '.',
time.sleep(1)
time.sleep(1)
print "Sent!"
smtpobj.quit()
Email providers and their SMTP servers
Gmail smtp.gmail.com Outlook.com/Hotmail.com smtp-mail.outlook.com Yahoo mail smtp.mail.yahoo.com AT&T smpt.mail.att.net (port 465) Comcast smtp.comcast.net Verizon smtp.verizon.net (port 465)
In the 3rd and 105th line, all necessary libraries are imported. 5th line is the menu that will ask the user when he/she wants to send mail. For convenience "morning", "afternoon", "evening" and "night" options are given. From line 11 to 90 all the menus are defined. The datetime library is used along with necessary parameters to format and displayed time and date. Line 1 to 101 is the scheduling mail part which is followed by sending mail using SMTP.
On line 106 SMTP object is created and gmail SMTP server is passed with out port 587. Out port is 587 for all except for AT&T and Verizon which is 465 as mentioned. On line 120 .ehlo() is like sending a "hello" message to gmail and if reply is positive (i.e an interger) then we may proceed. On line 113 .starttls() is used to start encryption for safety issues.
On line 120 getpass is a very nice function. It is used to accept passwords in script in hidden mode i.e your password will not be visible when you are typing it in while running the script. Line 124 - 126 asks you the details about the mail: mail id of receiver, subject of mail and body of mail.
On line 133, we calculate time difference between the: time that user decided to send the mail and present time. This is done so that we can make the script sleep for that time and send it after that. On line 145 SMTP sever is quit by passing the .quit() function.
Web Scraping In Python
Web scraping (web harvesting or web data extraction) is a computer software technique of extracting information from websites.
There are several modules that make it easy to scrape web pages in Python.
- Webbrowser: It comes with Python and opens a browser to a specific page.
- Requests: Downloads files and web pages from the Internet.
- Beautiful Soup: To parses HTML, the format that web pages are written in.
- Selenium: It can launch and control a web browser. It is able to fill in forms and simulate mouse clicks in browser and other interesting stuff.
I wrote a small script for a friend of mine last month. His team is into making small planes. They collect their data(like battery type, motor manufacturer, propeller type etc.) for their models from this german website.
So on the site there are about 60 battery types, 172 controller types, 22 propeller types, many different motor manufacturers and each manufacturer has different motor types. All this data appears in form of drop down menu. His objective is to get maximum thrust for input power <= 1000W.
Now it is not humanly possible to try every possible combination to get the desired result. Here comes the role of web scraping: extracting and manipulating content of a website to obtain desired result.
My web scraping code repository on git.
Web scraping (web harvesting or web data extraction) is a computer software technique of extracting information from websites.
There are several modules that make it easy to scrape web pages in Python.
Extension Modules in Python
Python is on the order of between 10 and 100 times slower than C, C++, FORTRAN when doing any serious number crunching.
There are many reasons for this:
- Python is interpreted
- Python has no primitives, everything including the builtin types are objects
- Python list can hold objects of different type, so each entry has to store additional data about its type.
But these are no reason to ignore Python though. A lot of software doesn't require much time or memory even with the 100 time slowness factor. Development cost is where Python wins with the simple and concise style. This improvement on development cost often outweighs the cost of additional cpu and memory resources.
It is quite easy to add new built-in modules to Python, if you know how to program in C. Such extension modules can do two things that cant be done directly in python:
- They can implement new built-in object types
- They can call C library functions and system calls
- This improves python's computational speed many folds.
My extension modules code repository on git.
Subscribe to:
Posts
(
Atom
)
Posts
