Postings by Samuel Williams

trenni-sanitize - a fast markup sanitizer

Hi - I've been processing a lot of HTML content with the intent to extract
either text summaries or sanitise it for further use. Frustrated primarily
by the existing sanitize gem due to it's performance and nokogiri's
handling of namespaces, I have released trenni-sanitize - a gem which uses
a native parser to sanitize general markup style input and runs about 50x
faster :)

<a href="" title=""></a>

Kind regards,

TracePoint line coverage


I'm playing around with TracePoint API and code coverage using :line event.

I'd like to be able to generate a percentage coverage report for a given
file. While I can easily see what lines are executed, I can't easily
determine what lines are not executed.

What I need, I think, is a tracepoint which triggers for eval/load/require,
which allows me to determine the initial coverage of the file.

Good library for command pipes in Ruby?


I tried looking for a library that lets me write (lazy) command line pipes
in Ruby but came up short.

I would like to make a function that invokes a executable to transform an
object into text and either saves it to a file or dumps it out.

Ideally, I could write it like this:

def pipeline
return self.to_text >> Process.spawn("dot")

Ideally, this pipeline is not invoked until the user does the following:

pipeline.write(path) //writes to file

or //returns String

Does something like this exist already?


async & async-io 1.0.0

Hello Friends,

I recently released async and async-io v1.0.0

<a href="" title=""></a>
<a href="" title=""></a>

- These gems provide zero-overhead syntax for asynchronous IO.
- Servers run inside a reactor task which can be started and stopped
- Very low overhead per IO operation.
- Scales to multiple CPU cores using containers
(<a href="" title=""></a>)
- Easy standardised testing using <a href="" title=""></a>

As already mentioned, there is DNS server available:
<a href="" title=""></a> and also a rack

RubyDNS 2.0.0 released.

RubyDNS provides a Ruby DSL for making a DNS server. It's trivial, for
example, to hook up DNS TXT requests to Wikipedia.

This release builds on several new gems for asynchronous IO including:

- async 1.0.0 (asynchronous events/timers using the reactor pattern & fibers)
- async-io 1.0.0
- async-dns 1.0.0

RubyDNS 1.x was built on top of Celluloid, while RubyDNS 2.0 is built
on top the above gems. With the utmost admiration and respect:
Celluloid's design made it hard to use in practice.

async-http -> falcon

I've quickly hacked together a rack compliant server using async-http
called falcon

<a href="" title=""></a>

It's completely alpha, but may be of interest to some who have been
following this.

Feel free to have a hack at it if it's something you are interested
in. It's currently using a multi-threaded model but it's simple to
choose another.

Unsubscribe: <>

async-http vs puma

I've got a local rack app which is a simple disk/file based wiki with
no caching whatsoever.

async-http hits 100,000 req/s

I've been working on a stack of gems (async, async-io and async-http)
as a proof of concept that Ruby can do fast async networking similar
to Node - and perhaps even better in some cases.

I have finally got async-http to the point where, as a
proof-of-concept, I think it's validated at least part of the above

<a href="" title=""></a>
<a href="" title=""></a>
<a href="" title=""></a>

On my desktop, I can get a nominal throughput of between 30,000 req/s
to 100,000 req/s with 4 cores/8 processes.

cloudflare (API) && cloudflare-dns-update gems

I've just released updated versions of these two gems.

cloudflare provides a RestClient::Resource based API for accessing Cloudflare.

On top of that I've built cloudflare-dns-update which uses tty-prompt
and samovar for an interactive command-line. It's pretty cool - video
here: <a href="" title=""></a>

Kind regards,

Unsubscribe: <>

Testing gem dependencies

I am working on a gem rb-inotify (let's call A from now on) which is a
dependency of listen (call B).

B depends on A.

A doesn't have many specs right now, behaviour is defined primarily by
correct functioning of B which has a lot of specs.

I want to run specs on travis for A (it has a few and we would add
more in the future), but I also want to run all of the specs for B

Before i have a go trying to make this work, does anyone have any


Unsubscribe: <>

build-dependency and build-uri

Build::Dependency is a dependency resolution algorithm. It's primarily
used in Teapot, primarily a C++ build tool (but it's completely
generic). However, I've decided to extract it, because I've also been
using it in a tool for deploying Arch Linux virtual machines. In this
case, it reads the PKGBUILD files and can construct a dependency
graph. It doesn't support versioned constraints (although it probably
could) as they are not useful in teapot.


Hi Everyone,

I've been slowly trying to improve facets

<a href="" title=""></a>

It's a pretty awesome gem, but it also needs a lot of work.

I was wondering if anyone else would be willing to help with the next release.

The goal for 4.0 is to really be a 2.1+ release, with all
functionality removed which is either a/ already supported natively by
ruby or b/ obscure to the point of being incorrect.

Kind regards,

Unsubscribe: <>

RubyDNS 2.0.0-rc1

Hi Everyone,

Perhaps this is a bit premature but I'm pretty excited to be updating RubyDNS.

<a href="" title=""></a>

In order to get to this point, I built a new asynchronous IO framework for Ruby:

<a href="" title=""></a>

`Async::Reactor` implements the reactor pattern and basically takes
all the good parts of `celluloid-io`.

Utopia - a rack based web framework

Hi Everyone,

I've just released the next major version of Utopia, source available
at <a href="" title=""></a>

It's a content-centric web framework for Ruby. It's fast (native ragel
parsers) multi-threaded (concurrent-ruby for internal caches,
rack-freeze for safety) and it's used in production for several years

Gem with native C for MRI, pure ruby fallback, how to compile?


I can't seem to figure out how this is supposed to work.

Here is my extconf.rb which is currently working:

<a href="" title=""></a>

But, I'm afraid it's a huge hack.

How is this supposed to work?

I should build package. On MRI, when installing, it should compile C

String capacity

If I know that a string is going to be at least X bytes in my native
extension, is there some way to set the initial capacity to avoid
memory allocations?

I see rb_str_tmp_new but I can't do that with an encoding.. could I just use

VALUE str = rb_str_tmp_new(X);
rb_enc_associate(str, enc);


Unsubscribe: <>

rb_str_new with custom subclass of rb_cString

Is it possible to construct a string with a custom subclass or replace the
klass pointer of a VALUE in an existing one?

I see rb_str_new_with_class but it actually takes an object and uses its
class rather than a class directly which might work but is a bit of a hack.
Ideally would be able to change the class of the object... but perhaps
that's asking too much?


ffi-clang for Ruby

Hi Everyone,

After the release of LLVM 4.0, I wanted to get the Ruby FFI gem
updated and working. It's been a long time since it's last gem

It can do some pretty awesome things (obviously most credit goes to

Merging multiple aliases files together

I don't know the best way to achieve this.

I have some "user" packages which install various users on a linux
system. I want the setup to be entirely automated, and as part of this
I want to set up some mail aliases for each user, and add their alias
to the root alias. Essentially, the following represents one user:

root: bob
bob: <a href="mailto: ... at example dot com"> ... at example dot com</a>

I have 0 or more of these user packages, and I want to, some how,
merge all these aliases together.

Best way to call Integer on temporary?

I like to use Float, Integer and String to typecast data from user. It
raises an exception if the value was invalid for some reason.

However, it doesn't easily support optional values, e.g. Integer(nil)
fails too. So, I end up writing code like:

result = (tmp = x[:foo]) ? Integer(tmp) : nil

But that seems really ugly.

Is there a better way without just writing my own wrapper all the

Best way to handle Integer/Bignum/Fixnum


In my code, after migrating to 2.4.0, I see a lot of

gems/mapping-1.0.0/lib/mapping/object_model.rb:26: warning: constant
::Fixnum is deprecated
gems/mapping-1.0.0/lib/mapping/object_model.rb:26: warning: constant
::Bignum is deprecated

This code still works okay, but I was wondering, how should I write my
code now to not produce warnings with both 2.3 and 2.4?


Relaxo - git backed database with lightweight trasactional API

Hello Friends.

I recently published Relaxo 1.0. It's a document storage engine
written on top of libgit2/rugged.

<a href="" title=""></a>

Relaxo exposes the underlying data model of git deliberately. It's
essentially a persistent data structure with algorithms implemented in
C. The relaxo gem is only ~150 lines of code. It's standing on the
shoulders of giants.

The nice thing about libgit2 is that it has some provisions for a
backend storage other than a file-system.

Trenni - a gem for fast templating and parsing


I've recently finished implementing upgrades to Trenni:
<a href="" title=""></a>

It now includes C parsers generated by Ragel to improve both
performance and correctness.

It's faster than Nokogiri and ERB which was exciting.

We use this in production from today and it reduces latency by up to
50% for some pages compared to previous StringScanner/Regex parser.

rb_funcall and explicit method cache

I'm implementing some code that calls the same method multiple times
from C. I'd like to cache the method to avoid costly method lookup. Is
this possible?

Something like

METHOD * m = rb_method_lookup(object, rb_intern("doit"));
rb_invoke(m, args...)

Is any such thing possible?


Unsubscribe: <>
<> what is default encoding?


When preparing for immutable strings by default, I changed my code from

buffer = ""


buffer =

However, an unintended side effect of this was the encoding:

2.3.1 :001 > "".encoding
=> #<Encoding:UTF-8>
2.3.1 :002 >
=> #<Encoding:ASCII-8BIT>

I'm replacing `""` with ` Encoding::UTF_8)` but
wondering what exactly is the best practice here going forward?

Kind regards,

Unsubscribe: <>

Is it possible to make your own String Encoding?

I'd like to define an encoding which is a derivative of UTF-8 but
includes HTML entities.

Is this possible? I don't see anything like, for example.

Unsubscribe: <>



I feel like this should work.

$ URI.parse("/base/uri") + URI.parse("relative")
URI::BadURIError: both URI are relative

The result should be URI with path = "/base/relative".

What do you think? Is this a bug? Or is it designed like this on purpose?

Kind regards,

Unsubscribe: <>

Substitute one gem for another in Gemfile?

I'm developing a gem with an idea to replace an existing gem which I don't
control. The replacement gem provides a compatible API.

For my own project, of course I can just switch the gem i the Gemfile. But,
what about other gems which depend on the original gem that I want to

Is it possible to say - this gem dependency will now be provided by some
other gem?

~> operator


I sometimes write code in irb/pry and I want to send the result into
another function.

Something like

Then, it turns out I wanted to write


So I have to move the cursor all the way to the left and insert f and
perhaps some more brackets, etc.

I was wondering if we can discuss some new operator, like following:

a ~> b
defined as b(a)

Some other languages already have such an operator.

Ideas? Discussion?

Makes my life easier: ~> f

It could be called the "goes into" operator.

Fingerprint - general purpose data integrity tool.

Fingerprint is a general purpose data integrity tool that uses
cryptographic hashes to detect changes in files. Fingerprint scans a
directory tree and generates a small transcript file containing the names
and hashes of the files. This snapshot file can then be used to generate a
list of files that have been created, deleted, or modified.