Игорь Олемской — практические заметки по системному администрированию Linux CentOS

When Twitter was still in its early stages, you could track certain search terms in near-realtime via Jabber. It was quite popular and its performance degraded over time as more users signed up and began posting updates. Eventually, Twitter killed the jabber bot altogether. Many users have asked when it will return.

Well, it hasn't returned, but you can build your own replacement with ruby, a jabber account, and a few gems. While it won't do everything that the original jabber bot did, you can still track tweets mentioning certain terms very quickly.

Here's how to get started:

First, install the tweetstream and xmpp4r-simple gems:

gem install tweetstream xmpp4r-simple

Next, you'll need a jabber account. You'll probably want to make one for the exclusive use of your jabber bot. I chose to make up a quick account at ChatMask for mine.

The last step is to drop a copy of this script on your server:

#!/usr/bin/ruby
require 'rubygems'
require 'tweetstream/client'
require 'tweetstream/hash'
require 'tweetstream/status'
require 'tweetstream/user'
require 'tweetstream/daemon'
require 'xmpp4r-simple'
 
jabber = Jabber::Simple.new('jabberbot@yourjabberserver.com','jabberpassword')
 
tweets = TweetStream::Client.new(twitterusername,twitterpassword)
 
tweets.track('celtics','lakers','finals','nba') do |status, client|
  imtext = "#{status.user.screen_name}: #{status.text} \r\n"
  imtext += "[http://twitter.com/#{status.user.screen_name}/status/#{status.id}]"
  jabber.deliver("yourjabberusername@yourjabberserver.com",imtext)
end
 
jabber.disconnect

You'll want to be sure to fill in the following:

• your jabber bot's username and password
• the username and password for the twitter account that will monitor the stream
• the search terms you want to track
• the destination jabber account where the messages should be sent

Ensure that your jabber account has authorized the jabber bot's account so that you'll actually receive the messages. Also, Twitter is very strict with their streaming API tracking terms. It's a good idea to review their entire Streaming API documentation to ensure that you're not going to end up having your account temporarily or permanently blacklisted.

Once everything is ready to go, you can just run the script within GNU screen or via nohup. There's still a bit more error checking to do around jabber reconnections, but the script has run non-stop for well over two weeks at a time without a failure.

Reincarnation of Twitter's realtime XMPP search term tracking with ruby is a post from: Major Hayden's Racker Hacker blog.

c0b6ad7e-f251-11df-b20b-4040336e00ef

My curiosity is always piqued when I find new ways to manipulate command line output in simple ways. While working on a solution to parse /proc/mdstat output, I stumbled upon the paste utility.

The man page offers a very simple description of its features:

Write lines consisting of the sequentially corresponding lines from each FILE, separated by TABs, to standard output.

Here's an example of how it works. Let's say you want to parse some software raid output that looks like this:

# mdadm --brief --verbose --detail /dev/md0
ARRAY /dev/md0 level=raid1 num-devices=2 metadata=00.90 UUID=7bea4601:d5a02f5c:2da69848:3184a367
   devices=/dev/sda1,/dev/sdb1

It would be handy if we had both on one line as that would make it easier to parse with a script. Of course, you can do this with utilities like awk and tr, but paste makes it so much easier:

# mdadm --brief --verbose --detail /dev/md0 | paste - -
ARRAY /dev/md0 level=raid1 num-devices=2 metadata=00.90 UUID=7bea4601:d5a02f5c:2da69848:3184a367	   devices=/dev/sda1,/dev/sdb1

By default, paste uses tabs to separate the lines, but you can use the -d argument to specify any delimiter you like:

# mdadm --brief --verbose --detail /dev/md0 | paste -d"*" - -
ARRAY /dev/md0 level=raid1 num-devices=2 metadata=00.90 UUID=7bea4601:d5a02f5c:2da69848:3184a367*   devices=/dev/sda1,/dev/sdb1

Parsing mdadm output with paste is a post from: Major Hayden's Racker Hacker blog.

c0b6ad7e-f251-11df-b20b-4040336e00ef