Audio Rendering of STEM Content

Venkatesh Potluri and I have made the code used in the "Audio Rendering projects" publicly available !

Here is the link:
https://github.com/saikrishnar/AudioRenderingofSTEM

These are a series of projects that we undertook to render the STEM content in audio with an intention to assist the people with print disabilities, who face a lot of difficulties today to be able to even access such content, while pursuing STEM education is still a distant dream. It is surprising that even with such developments in technology, such people have to really struggle for education.

For example, consider something as simple as math equations. Mathematics, in its visual form, gives the reader a very high level of granularity in perceiving the equation (we get the concept of subscript and superscript by just glancing at an equation). Now, try to picture the situation of a visually impaired student. Yes, we can use a text to speech engine and make it read the equation. The point dies here itself coz as we all know, math in itself is quite complex and speaking an equation just like a normal sentence leaves the student with all the more confusion. In other words, the TTS systems today do not have such granularity as is available when we SEE an equation and it is beyond reason to try to comprehend something in mathematics without having such clarity, even for normal sighted people.

Through these projects, we tried to formulate a systematic mechanism to render such content in audio by applying several "hacks" on the current Text to speech, thereby enabling everyone access to STEM. Venkatesh is certainly a torch bearer himself on this path and has successfully finished his Masters thesis at IIIT-H. However, the real satisfaction came when we tested the techniques on the visually challenged people at LVPEI Hospital, Hyderabad and they were able to solve complex math equations by using these techniques, more importantly, on par with the normal sighted people (These normal sighted people are students of IIIT-H and are obviously STROONG at Math. The mere fact that the visually impaired people could match these students is a huge thing). As of now, we support Math equations, statistical data such as pie/bar charts and stoichiometric equations from organic chemistry.

Having said that, this is still an asymptotic case. Will be great if some kind of product can be materialized out of this, thus having a real and profound impact on the people who are in real need and have very little help!!

Scheduler Module for Multitasking in Veronica - A python based personal assistant

Hi,

This has been a hectic ( also full of life ) summer with a lot of tasks.

Although I'd prefer working on a single task with an undeterred focus, I had no option but to multi task given the significance of the works at hand. There was no escape. Uff ! Deadlines every alternate day and that was insanely fun.  However, these are unplanned, unintended and kill our productivity most of the times ( well, atleast for me ! ). I was itching myself to write a small python module for future situ like this, hence this post.  To brag,  this is the multitask - helping module of my favorite python based assistant, Veronica specifically for such busy days.

Basically, this is a scheduler which parses the user's Google Calendar and notifies the tasks thereby trying to co ordinate the work.

The figure below shows the simplistic overview of the steps involved.

Google has a clear python based tutorial explaining the procedure to use Calendar api. Just mentioning the steps here:

1. Create a new project on the Google Developer Console
2. Enable the Calendars API.
3. Create new client ID for standalone app and download the JSON.


Rest is a basic parser in python which extracts the time and the event from the Calendar output. The script then makes a trivial hour based comparison and notifies the user of the event. I'm using pynotify to display the message and espeak to provide speech feedback to the user.

Display Block
########################################################

def sendmessage(title, message):
    pynotify.init("Test")
    notice = pynotify.Notification(title, message)
    notice.show()
    return

###############################################################

Here's a picture of the script at work.

The code is shared on github. Please feel free to play around !

Linux based Personal Music Player using Python

Dear readership

It’s a softer blog today, like dark chocolate for dessert. To those who know, and to those who must find out : Our best friend in music has tragically orphaned us on the internet. We wail out loud and beat our chests, but in code. This post is no epitaph, but an insignificant reverberation in the just-created hollow. We disclaim : We write, as always, for non-coders and coders alike.

First, let’s prepare a mis-en-place of modules we’re going to need. PyAudio has long served the Speech Processing community for storing and splitting the wave files. It’s the obvious invited. Second in line is PyQT4, that runs the baseline for all that you can see and prod. (The GUI, yeah, you got it). The sycophants include modules like “time”, “wave”, “threading” and “sys”. Look below to get a complete list. We will go over the functions of each as you read.

Towards Step 1. 

Our first job is to define the backend function of playing and stopping songs. You instantiate PyAudio using this :

p = pyaudio.PyAudio()

We then proceed to define a function that loops over a folder, and plays music on its own. All you need is a directory with a set of wave files(extension : .wav). 

The function instantiates the pyaudio module. The flag defined above is sorta important, to make sure your flitting between music files, keeping track, and pausing is controlled.  

We move over to the classes are defined functions, one after the other. The first is a conventional initialization of the GUI function. Not much syntax, not much theory there. Easy enough.

The initUI function is where the real magic rests. 

First, you define 3 buttons: Play, Next and Stop. Use button-name.move( ) to position them on your window-to-appear. Divide the space, it takes a couple tries, but you’ll get around to it. 

The ‘Exit’ button lets you abort the program when it misbehaves. 

**********************************

def initUI(self): btn1 = QtGui.QPushButton("Play", self) btn1.resize(btn1.sizeHint()) btn1.move(50, 30) btn2 = QtGui.QPushButton("Stop", self) btn2.resize(btn2.sizeHint()) btn2.move(150, 30) btn3 = QtGui.QPushButton("Next", self) btn3.resize(btn3.sizeHint()) btn3.move(250, 30) btn4 = QtGui.QPushButton("Exit", self) btn4.resize(btn3.sizeHint()) btn4.move(350, 30)

***************************************

Always handy, we thought. Notice that innocuous “count” there. Let that be, we’ll talk about it next week. As of now, it’s marks a counter that makes sure which wavfile is being played.

Ok.. let’s now get into the actual space between the touch and the heartstring. In his modesty, my co-author calls it the buttonClicked function. I like to keep it that way, for my kinda people. In 2 simple lines : press the “Play”, run the baseline function that plays music in the wavefiles. It is here that the connection between the button on the GUI and your backend communicate and act. Essentially, button clicked. 

***********************************************************

btn1.clicked.connect(lambda: self.buttonClicked(count)) btn2.clicked.connect(lambda: self.buttonClicked(count)) btn3.clicked.connect(lambda: self.buttonClicked(count)) btn4.clicked.connect(QtCore.QCoreApplication.instance().quit)

************************************************************

The assignment of flags here is glaringly important. (ask me, I spent the afternoon on it) This is what will let the play_files decide what to play and when.  For when you press “Stop”, the flag that is set to 1 will render play_files inactive. The Next button is not much different, you run the playlist function again, just that your count index is now the next file in the playlist.

*********************************************************

def buttonClicked(self, count): sender = self.sender() #print sender if sender.text() == 'Play': self.statusBar().showMessage('Yipee') flag = 0 p = pyaudio.PyAudio() #t = multiprocessing.Process(target=play_file()) #t1.daemon = False #threads.append(t) #t.start() play_file(flag,count) elif sender.text() == 'Stop': flag = 1 self.statusBar().showMessage('Stopping :(') #t2 = threading.Thread(target=play_file(flag)) play_file(flag, count) elif sender.text() == 'Next': self.statusBar().showMessage('Yipee') flag = 0 count = count + 1 p = pyaudio.PyAudio() #t = multiprocessing.Process(target=play_file()) #t1.daemon = False #threads.append(t) #t.start() play_file(flag,count) else: self.statusBar().showMessage(sender.text() + ' was pressed') stream.stop_stream()

**********************************************************

That’s all folks! All you got to do is put the classes and functions together. Call them one by one and sing along.

The code is made opensource for you to try out and make changes as you wish. Navigate to the scripts folder and hit ./music_player.py!!

Desktop Live Score Board using Python to parse XML

Its IPL time !!! well.. to be honest,  somehow I wasn't too interested in this year's IPL, atleast before it started.  Few matches into it and the knocks by the B-Man from Chennai and Gayle made me think again. 

Having said that, this is quite a busy time which means no live streaming...well..at least some times.  Ya, cricinfo ..., but switching between tabs too isnt trivial, especially when coding a particular module on text editors.

So, I thought of writing a small snippet which can fetch the live scores onto the desktop. Luckily, Cricinfo provides RSS in XML format for non commercial purposes which can be leveraged for this.

So whats this XML????    Lets try to decipher !!!

XML is a basically a markup language that defines a set of rules for encoding documents in a format which is both human-readable and machine-readable. It is a textual data format with tremendous support for quite a lot human languages.There are three components in XML: Declaration, Markup and Content. 

To keep it simple, lets say that every XML document begins with a declaration section where the specifics about the document format  can be found. 

Every thing that's between the special characters '<' and '>' is markup and the rest is the content. Having said that, markup is also specified between  '&' and ';', but lets not bother much about it now.

Markup and Content make up an XML Element. An XML document can be viewed as an accumulation of such XML elements.

Once we get  this basic understanding, the livescores file in the xml format can be parsed using Python for our task.  I've written a small script which does the same and pops up a desktop notification once every minute about the live scores. 

Example Rendering :

To run the script, navigate to the folder via teminal and type:

python scoreboard.py 'Number'

Here Number corresponds to the match you want the score for. 


Example:
Suppose the xml file looks this way:

<title>
Sri Lanka Air Force Sports Club 215/10 v Galle Cricket Club 31/10 & 278/7 *
</title>
<link>
http://www.cricinfo.com/ci/engine/match/859469.html?CMP=OTC-RSS
</link>
<description>
Sri Lanka Air Force Sports Club 215/10 v Galle Cricket Club 31/10 & 278/7 *
</description>
<guid>
http://www.cricinfo.com/ci/engine/match/859469.html
</guid>
</item>
<item>
<title>Hampshire v Sussex</title>
<link>
http://www.cricinfo.com/ci/engine/match/804159.html?CMP=OTC-RSS
</link>
<description>Hampshire v Sussex</description>
<guid>
http://www.cricinfo.com/ci/engine/match/804159.html
</guid>
</item>

Then

Number = 1 corresponds to the match between Srilankan Airforce vs Galle Cricket Club. 
Number = 2 corresponds to the match between Hampshire and Sussex
Feel free to play around with the script. 

Possible  Causes for Error:

1)  No module named bs4:

The script uses  Beautiful Soup to parse the XML, the absence of which is the cause of this error. 

BS can be installed via terminal using the command:
pip install beautifulsoup4

1)  No module named requests:

Requests can be installed via terminal using the command:
pip install requests

P.S. Use sudo in case of permission error
sudo pip install beautifulsoup4 
sudo pip install requests