Игорь Олемской — практические заметки по системному администрированию Linux CentOS

I'm using SELinux more often now on my Fedora 15 installations and I came up against a peculiar issue today on a new server. My PHP installation is configured to store its sessions in memcached and I brought over some working configurations from another server. However, each time I accessed a page which tried to initiate a session, the page load would hang for about a minute and I'd find this in my apache error logs:

[Thu Sep 08 03:23:40 2011] [error] [client 11.22.33.44] PHP Warning:
Unknown: Failed to write session data (memcached). Please verify that
the current setting of session.save_path is correct (127.0.0.1:11211)
in Unknown on line 0

I ran through my usual list of checks:

• netstat showed memcached bound to the correct ports/interfaces
• memcached was running and I could reach it via telnet
• memcached-tool could connect and pull stats from memcached
• double-checked my php.ini
• tested memcached connectivity via a PHP and ruby script — they worked

Even after all that, I still couldn't figure out what was wrong. I ran strace on memcached while I ran a curl against the page which creates a session and I found something significant — memcached wasn't seeing any connections whatsoever at that time. A quick check of the lo interface with tcpdump showed the same result. Just before I threw a chair, I remembered one thing:

SELinux.

A quick check for AVC denials showed the problem:

# aureport --avc | tail -n 1
4021. 09/08/2011 03:23:38 httpd system_u:system_r:httpd_t:s0 42 tcp_socket name_connect system_u:object_r:memcache_port_t:s0 denied 31536

I'm far from being a guru on SELinux, so I leaned on audit2allow for help:

# grep memcache /var/log/audit/audit.log | audit2allow
 
#============= httpd_t ==============
#!!!! This avc can be allowed using one of the these booleans:
#     httpd_can_network_relay, httpd_can_network_memcache, httpd_can_network_connect
 
allow httpd_t memcache_port_t:tcp_socket name_connect;

The boolean we're looking for is httpd_can_network_memcache. Flipping the boolean can be done in a snap:

# setsebool -P httpd_can_network_memcache 1
# getsebool httpd_can_network_memcache
httpd_can_network_memcache --> on

After adjusting the boolean, apache was able to make connections to memcached without a hitch. My page which created sessions loaded quickly and I could see data being stored in memcached. If you want to check the status of all of the apache-related SELinux booleans, just use getsebool:

# getsebool -a | grep httpd | grep off$
allow_httpd_anon_write --> off
allow_httpd_mod_auth_ntlm_winbind --> off
allow_httpd_mod_auth_pam --> off
allow_httpd_sys_script_anon_write --> off
httpd_can_check_spam --> off
httpd_can_network_connect_cobbler --> off
httpd_can_network_connect_db --> off
httpd_can_network_relay --> off
httpd_can_sendmail --> off
httpd_dbus_avahi --> off
httpd_enable_ftp_server --> off
httpd_enable_homedirs --> off
httpd_execmem --> off
httpd_read_user_content --> off
httpd_setrlimit --> off
httpd_ssi_exec --> off
httpd_tmp_exec --> off
httpd_unified --> off
httpd_use_cifs --> off
httpd_use_gpg --> off
httpd_use_nfs --> off

If you're interested in SELinux, a good way to get your feet wet is to head over to the CentOS Wiki and review their SELinux Howtos

Getting apache, PHP, and memcached working with SELinux is a post from: Major Hayden's Racker Hacker blog.

Thanks for following the blog via the RSS feed. Please don't copy my posts or quote portions of them without attribution.

While trying to get command-t VIM plugin installed I noticed that the default vim-7.0.x installed from CentOS Base repository does not come compiled with ruby support and needed to be upgraded to version 7.2.x:

Notice «-ruby» which states that it is without ruby support.

Once the rpmbuild environment is setup, install the source rpm:

Edit the vim.spec file, line 496 and remove perl-devel from the requirement list.

Then build rpm binary:

Install the rpms via:

read more

When Twitter was still in its early stages, you could track certain search terms in near-realtime via Jabber. It was quite popular and its performance degraded over time as more users signed up and began posting updates. Eventually, Twitter killed the jabber bot altogether. Many users have asked when it will return.

Well, it hasn't returned, but you can build your own replacement with ruby, a jabber account, and a few gems. While it won't do everything that the original jabber bot did, you can still track tweets mentioning certain terms very quickly.

Here's how to get started:

First, install the tweetstream and xmpp4r-simple gems:

gem install tweetstream xmpp4r-simple

Next, you'll need a jabber account. You'll probably want to make one for the exclusive use of your jabber bot. I chose to make up a quick account at ChatMask for mine.

The last step is to drop a copy of this script on your server:

#!/usr/bin/ruby
require 'rubygems'
require 'tweetstream/client'
require 'tweetstream/hash'
require 'tweetstream/status'
require 'tweetstream/user'
require 'tweetstream/daemon'
require 'xmpp4r-simple'
 
jabber = Jabber::Simple.new('jabberbot@yourjabberserver.com','jabberpassword')
 
tweets = TweetStream::Client.new(twitterusername,twitterpassword)
 
tweets.track('celtics','lakers','finals','nba') do |status, client|
  imtext = "#{status.user.screen_name}: #{status.text} \r\n"
  imtext += "[http://twitter.com/#{status.user.screen_name}/status/#{status.id}]"
  jabber.deliver("yourjabberusername@yourjabberserver.com",imtext)
end
 
jabber.disconnect

You'll want to be sure to fill in the following:

• your jabber bot's username and password
• the username and password for the twitter account that will monitor the stream
• the search terms you want to track
• the destination jabber account where the messages should be sent

Ensure that your jabber account has authorized the jabber bot's account so that you'll actually receive the messages. Also, Twitter is very strict with their streaming API tracking terms. It's a good idea to review their entire Streaming API documentation to ensure that you're not going to end up having your account temporarily or permanently blacklisted.

Once everything is ready to go, you can just run the script within GNU screen or via nohup. There's still a bit more error checking to do around jabber reconnections, but the script has run non-stop for well over two weeks at a time without a failure.

Reincarnation of Twitter's realtime XMPP search term tracking with ruby is a post from: Major Hayden's Racker Hacker blog.

c0b6ad7e-f251-11df-b20b-4040336e00ef

It certainly shouldn't be difficult, but I always have a tough time with OAuth. Twitter is dropping support for basic authentication on June 30th, 2010. I have some automated Twitter bots that need an upgrade, so I've been working on a quick solution to generate tokens for my scripts.

I formulated a pretty simple script using John Nunemaker's twitter gem that will get it done manually for any scripts you have that read from or update Twitter:

#!/usr/bin/ruby
require 'rubygems'
require 'twitter'
 
# These credentials are specific to your *application* and not your *user*
# Get these credentials from Twitter directly: http://twitter.com/apps
application_token = '[this should be the shorter one]'
application_secret = '[this should be the longer one]'
 
oauth = Twitter::OAuth.new(application_token,application_secret)
 
request_token = oauth.request_token.token
request_secret = oauth.request_token.secret
puts "Request token => #{request_token}"
puts "Request secret => #{request_secret}"
puts "Authentication URL => #{oauth.request_token.authorize_url}"
 
print "Provide the PIN that Twitter gave you here: "
pin = gets.chomp
 
oauth.authorize_from_request(request_token,request_secret,pin)
access_token = oauth.access_token.token
access_secret = oauth.access_token.secret
puts "Access token => #{oauth.access_token.token}"
puts "Access secret => #{oauth.access_token.secret}"
 
oauth.authorize_from_access(access_token, access_secret)
twitter = Twitter::Base.new(oauth)
puts twitter.friends_timeline(:count => 1)

When you run the script, it will give you a request token, request secret and a URL to visit. When you access the URL, you'll be given a PIN. Type the PIN into the prompt and you'll get your access token and secret. This is what you can use to continue authenticating with Twitter, so be sure to save the access token and secret.

From then on, you should be able to login with a script like this:

#!/usr/bin/ruby
require 'rubygems'
require 'twitter'
 
application_token = '[this should be the shorter one]'
application_secret = '[this should be the longer one]'
 
oauth = Twitter::OAuth.new(application_token,application_secret)
 
oauth.authorize_from_access(access_token, access_secret)
twitter = Twitter::Base.new(oauth)
puts twitter.friends_timeline(:count => 1)

I hope this helps!

Idiot's guide to OAuth logins for Twitter is a post from: Major Hayden's Racker Hacker blog.

c0b6ad7e-f251-11df-b20b-4040336e00ef

Делаю очередной проект на Ruby on Rails. Как обычно, в миграциях понадобились foreign keys на уровне БД. Окинул взглядом все плагины, которые смог найти в google и на github.com. Ни один из них не умеет делать FK, используя ActiveRecord::ConnectionAdapters::Table#references и ActiveRecord::ConnectionAdapters::TableDefinition#references.

Поэтому сел и написал свой плагин: active_record_foreign_keys.

Установка:

Rails::Initializer.run do |config|
  ...
  config.gem "active_record_foreign_keys", :source => "http://gemcutter.org"
  ...
end

$ rake gems:install

Использование:

def self.up
  # create reference table
  create_table :users do |t|
  end
  # create referencing table
  create_table :a_examples do |t|
    t.references :user, :foreign_key => true
  end
  # or
  create_table :b_examples do |t|
    t.references :user, :foreign_key => {  n_update => :cascade,  n_delete => :restrict }
  end
  # or
  create_table :c_examples do |t|
  end
  add_foreign_key :c_examples, :user_id, :users, :id,  n_update => :no_action,  n_delete => :set_null
  # or change existing table
  change_table :d_examples do |t|
    t.references :user, :foreign_key => true
  end
end
def self.down
  # remove constraint
  remove_foreign_key :examples, :user_id, :users, :id
end

Плагин тестировался только под PostgreSQL, но по идее должен работать и под MySQL, и под Sqlite.

Прежде всего необходимо установить сам ruby

yum install ruby ruby-devel ruby-libs ruby-irb ruby-rdoc ruby-mysql rubygems

Так же понадобится компилятор и заголовки ImageMagick

yum install gcc make ImageMagick-devel

Для корректной работы не помешает обновить gem

gem update --system
gem update

Для компиляции rmagick потребуются шрифты, для установки которых потребуется rpmbuild и cabextract

yum install rpm-build cabextract

В /usr/src/redhat/BUILD/msttcorefonts/ необходимо загрузить шрифты, которые доступны по ссылке http://sourceforge.net/project/showfiles.php?group_id=34153&release_id=105355 (нужно загрузить непосредственно .exe-файлы со шрифтами).

Загрузите в папку /usr/src/redhat/SPECS файл http://corefonts.sourceforge.net/msttcorefonts-2.0-1.spec и запустите

rpmbuild -ba msttcorefonts-2.0-1.spec

Теперь можно установить шрифты

rpm -Uhv /usr/src/redhat/RPMS/noarch/msttcorefonts-2.0-1.noarch.rpm

Для установки rmagick, потребуется сделать symlink

ln -s /usr/share/fonts/msttcorefonts /usr/share/fonts/default/TrueType

А теперь можно смело устанавливать rmagick

gem install rmagick -v 1.15.17

Иногда нужно использовать валидацию для введенных через форму данных, но при этом эти данные никакого отношения к базе данных не имеют. Например, данные формы обратной связи.

Можно сэмулировать стандартную модель использую такой базовый класс-заглушку:

module ActiveRecord
  class Model
    def id; nil; end
    def new_record?; true; end
    def save; nil; end
    def save!; nil; end
    class << self
      def human_name(options = {})
        defaults = self_and_descendants_from_active_record.map do |klass|
          :"#{klass.name.underscore}"
        end
        defaults << self.name.humanize
        I18n.translate(defaults.shift, {:scope => [:activerecord, :models], :count => 1, :default => defaults}.merge(options))
      end
      def human_attribute_name(attribute_key_name, options = {})
        defaults = self_and_descendants_from_active_record.map do |klass|
          :"#{klass.name.underscore}.#{attribute_key_name}"
        end
        defaults << options[:default] if options[:default]
        defaults.flatten!
        defaults << attribute_key_name.humanize
        options[:count] ||= 1
        I18n.translate(defaults.shift, options.merge(:default => defaults, :scope => [:activerecord, :attributes]))
      end
      def self_and_descendants_from_active_record
        klass = self
        classes = [klass]
        while klass != klass.base_class
          classes << klass = klass.superclass
        end
        classes
      rescue
        [self]
      end
    end
    def initialize(params = nil)
      unless params.nil?
        params.each do |k,v|
          self.send("#{k}=", v)
        end
      end
    end
    include ActiveRecord::Validations
  end
end

Класс псевдомодели для формы обратной связи будет выглядить так:

class Message < ActiveRecord::Model
  attr_accessor :name
  attr_accessor :email
  attr_accessor :subject
  attr_accessor :body
  validates_presence_of :name
  validates_length_of :name, :within => 2..16, :allow_blank => true
  validates_presence_of :email
  validates_length_of :email, :within => 6..32, :allow_blank => true
  validates_format_of :email, :with => Authlogic::Regex.email, :allow_blank => true
  validates_presence_of :subject
  validates_length_of :subject, :within => 2..32, :allow_blank => true
  validates_presence_of :body
  validates_length_of :body, :within => 5..4096, :allow_blank => true
  def send_to(recipient)
    # send message to recipient
  end
end

Такую псевдомодель можно использовать в шаблонах с хелпером form_for как обычную. Интернационализация сообщений об ошибках работает так же, как если бы это была обычная модель ActiveRecord::Base.

Пример использования:

class ContactsController < ApplicationController
  def new
    @message = Message.new
  end
  def create
    @message = Message.new(params[:message])
    if @message.valid?
      @message.send_to(configatron.contacts.email)
      flash[:notice] = "Сообщение успешно отправлено"
      redirect_to new_contact_path
    else
      render :action => "new"
    end
  end
end