POE.... AnyEvent....3...2..1.. FIGHT!

(Uhh, sorry my english is not really good)

I din't really work with AnyEvent, but 1-2 weeks ago i worked through the Tutorial of AnyEvent. My First impression was that it is really cool. It is very easy, and after a couple of hours you can already use it to write usefull stuff.

I didn't really work with POE at the moment but read a lot about it and i read the chapters in the Book "Advanced Perl Programming 2.nd Edition). But still i found POE hard to learn and to understand.

I think there are some points that are better in AnyEvent.

1) With AnyEvent you can write an Event Based Module. But because AnyEvent uses the Event Loop that is already loaded, or the event loop loads an event loop itself, you can write one Module and use it in POE, Gtk, Tk, Glib and all the others Event Loop that AnyEvent supported. Compared to POE you write an module for POE and you can only use it in POE, not in another Event Loop.

2) Speed
AnyEvent itself supports a lot of Event Loops. The "best" and fastes Event Loop is "EV". This is a C based Event Loop. If "EV" is avaible AnyEvent uses this. And this Event Loop is really fast. Even other Event Loops like Glib or Event are faster compared to POE.

If you need some speed you can just install one of this Event Loop and your Performance increases. But this only works if you don't already use another Event Loop. If you load POE before then AnyEvent will use POE.

3) Just a Module
The really cool thing is that you can write just a "normal" module. The user of the module don't sees that there is a Event Loop behind the scenes.

A user can just load it as a normal module in his (not event based) application. The user just uses the API and the module does behind the scenes all the Event based stuff.

This is not something you can have with POE. If you write a POE based module, your user must use POE.

4) Coro
I didn't use it at the moment, but there is a module named Coro. From the documentation this seems to be real threads not the built in pseudo threads in Perl.

This threads with Coro are fast, from the module description

> And Coro's threads are fast, too: disabling the Windows process
> emulation code in your perl and using Coro can easily result in
> a two to four times speed increase for your programs. A parallel
> matrix multiplication benchmark runs over 300 times faster on a
> single core than perl's pseudo-threads on a quad core using all
> four cores.

Okay, but what has Coro to do with AnyEvent?

Coro is from the same developer that creates AnyEvent and he gives support for Coro in AnyEvent.

The Module is: Coro::AnyEvent

I didn't use/read it at the moment. But the combination AnyEvent + Coro looks like a really cool thing. Real threads, and good performance in Perl.

And this is something that Perl or other modules doesn't provide at the moment.

Having used Coro, it really is pretty awesome. However, it is not *real* threads. What it is is "real" threads confined within one application (and within one thread of that app) and created/controlled entirely by Perl. Coro does not allow you to make use of multi-core processors the way *real* threads would, you still need those for that. However inside one actual thread it allows the program to behave in a DWIM fashion as far as concurrency goes.

Furthermore, the "controlled entirely by Perl" part means that while you have a lot of control, you also have a lot of responsibility. Unless you're dealing with a very simple scenario, you're forced to write a co-routine controller on your own, switch manually and balance everything properly, or you end up clobbering performance by switching too often or not switching often enough.

Also, a request to the author: Would it be possible to get an article or two that explains in a nutshell exactly what POE or AnyEvent are and do for someone who's never used them before?

I used to do a lot of POE, it works great, but it's a lot of boiler plate code to do anything. AnyEvent feels nimbler, less repeating of code.

OTOH, you have to build everything, there is a very limited number of modules that work with AnyEvent, so a lot of common jobs require you to build the server or client from scratch.

Let me add my vote for Coro. I used it to write a server daemon and it was heavenly. Since it uses green threads (non-preemptive) it allows you to program in a multi-threaded way without having to worry about locks or difficult race conditions.

What Coro does is manage each thread in a global context--that is, each thread that is waiting on I/O gets put into the main select loop. So even though it looks like a threading extension, at its heart it really is a event framework.

You can think about it as an event framework turned inside out--instead of having to register for callbacks on a file/socket, you just do the read or write to that file/socket and Coro makes it asynchronous for you. It's inside out because now all your waiting is done deep inside the code blocks instead of managing callbacks at the top level.

I was discussing this on irc://irc.perl.org/win32 the other day (and I'm glad the latest AnyEvent update makes it pass testing on Win32 again). POE has the modules, certainly, but not the speed, simplicity or nimbleness of AnyEvent. I used to be the biggest POE fan until I had to use it for real work and now I'm a complete AnyEvent convert. Yes, you can use POE as your backend event loop in AnyEvent, but it is slower than using Event, EV or even Glib. You could probably use Object::Event to create AnyEvent/Moose sugar, like MooseX::POE has (I'm not volunteering). AnyEvent is also used by IO::Lambda (if you think Coro is an inside-out event loop, you should take that out for a spin).

And yes, Coro is awesome, it uses co-operative threading with multiple perl interpreters (maybe someone should write POE::Loop::Coro, and no, not volunteering for that either). One of the nice things Coro has is Coro::Select which overrides Perl core's select call with one that co-operates with Coro, allowing you to use certain modules (like Net::DBus) that block on select in a non-blocking fashion with Coro. Also making Coro worth having is Continuity, a module for web application which lets you create stateful web applications (prompt user with html form, wait for result, send next page, all handled sequentially).

Never had the good fortune (time or energy) to dig into the wonderful world of Poe. But if we already have Catalyst, why bother with Poe? What are the differences between the two, along with the advantages versus disadvantages?

@kiffin

You can call POE a "multi-tasking" framework. To do many things in parallel. Catalyst itself is a web framework.

POE and Catalyst are complete differnt things. Beause of this you can't say POE/Catalyst has this or that (dis)advantages over the other.

I have used POE for quite some time and loved it... unless I came across AnyEvent! POE works well, has a lot of modules on CPAN and no doubt is a winner in many areas, but... it's slow compared to AnyEvent, heavy weight (see RAM usage for example) and requires very specific programing convention which even after several years is still hard to get a good grip of. Also documentation for POE is IMHO not so good. For my latest project I used AnyEvent and it rocked! AnyEvent is fast, lightweight and more flexible than POE... and very very stable - on par with POE. The only bottleneck is that it doesn't have so many modules yet but I think that will change.

I'm a big fan of both projects but now leaning much more towards AnyEvent.

Actually, the truly new thing about AnyEvent is that it works as a client to other event loops (as some others already pointed out).

As for AnyEvent, combining it with POE/IO::Async etc. is not so difficult if you have Coro. Try this:

a) use the Event backend of POE (there *is* an EV backend,
but it seems to be unstable)
b) use AnyEvent wherever you like, which will automatically pick up EV (or Event, or POE, the latter is to be avoided)

while using Event or EV as POE backend will not really make POE much faster, it allows *other* parts of your process to become faster, by hooking directly into the faster event loop via AnyEvent for speed, avoiding the slow POE interface.

The effect is that a) and b) let you mix and match anyevent-based modules in a poe program, with no loss of speed (for anyevent) and no loss of richness (poe modules). i.e., use AnyEvent::HTTP if you like it's speed and/or simplicity, or use POE::Component::Whatever, simply because it exists and works.

So while POE, IO::Lambda, IO::Async and so on monopolise your process, you can combine _one_ of those together with AnyEvent modules, as long as they use a backend supported by AnyEvent (or are the backend supported by AnyEvent).

And then there is c), too:

c) run POE itself in a thread

This is entirely optional, but in theory, unless POE gets angry (it contains a lot of checks designed to make your life harder), you can just do:

async { POE::Kernel->run };

and forget about the poe mainloop and do your own stuff.

I haven't tried this, but it's likely possible to run POE and e.g. IO::Async in their own coro thread, as long as both use e.g. Event or EV as backend. Coro will multiplex between them, and being event-based framework, they are designed to block in the underlying event loop only, which is where coro will catch them.

So with AnyEvent, you can combine *any* AnyEvent modules with *one* traditional monopolising framework, and if you add Coro, you might even be able to combine multiple ones, all in the same process. Total freedom...

As for the person who said coro are not *real* threads, get a textbook and study it, or read wikipedia: you confuse threads with processes (being able to run in parallel is not a defining property of threads at all (shared address space is), and in fact, threads are much worse for multicore systems than processes, due to the extra synchronisation between cores required - threads were invented to speed up single core systems, for multi core systems, processes are much more efficient).

I thought this post is very valueable to share with that I translated it (including comments) to Korean and posted it on my own blog with source url. please tell me if it matters. thank you! :D

[Korean] http://amoc.tistory.com/17

Please help stop the spread of misinformation.

1) POE automatically detects and uses other event loops, as it has done for nearly a decade. See CPAN for a current list of POE::Loop modules.

Code that uses POE will run with little or no change in any event loop that POE supports. POE runs as a "client" to the native event loop, so native events are dispatched along with POE's.

2) Most of the differences you cite are based on an incorrect assumption that POE can't load other event loops.

POE's performance is very much dependent upon how it's used. IO::Lambda's benchmarks show that POE (using its default select loop) is between 2.6x and 8x slower than raw sockets using Perl's built-in select(). The difference depends whether an application uses POE's low-level select_read() method, the mid-range POE::Wheel abstractions, or the high-level POE::Component abstractions.

POE can be made faster in at least four ways. First, load a faster event loop. Second, consider replacing some of POE's plain Perl code with drop-in XS versions from CPAN. Third, consider publishing your own XS replacements if more speed is needed. You can reuse some of POE's tests, which are on the CPAN to help you develop extensions. Finally, POE's Perl code has ample room for improvements---contribute some.

Above all, profile and benchmark. Many bottlenecks are actually in third-party modules or application code. You can't solve them by optimizing POE.

3) POE doesn't prevent people from writing "normal" modules. In fact, there are more ways to use POE than many other Perl modules.

Take for example LWP::UserAgent::POE. This module is nearly identical to LWP::UserAgent, but it uses POE internally.

POE::Session subclasses alter the way POE calls application code. CPAN hosts several alternatives that change a lot of POE's look and feel.

There's also MooseX::POE, which provides a more contemporary way to use POE.

I've written at least three object frameworks that entirely change the way POE programs could be written. My latest one, POE::Stage, could use more input.

4) Someone who likes Coro and POE can write Coro::POE. Hop on irc.perl.org #poe if you have any questions.

@rocco: please _you_ stop spreading misinformation :(

POE can _load_ and _use_ other event modules, but it _cannot_ work besides them like AnyEvent does.

POE _cannot_ be used in an existing Tk or Gtk program. You can use Gtk as a backend to POE, and use Gtk from *within* a POE program only.

The program as a whole always needs to be a POE program. Even a single POE module "infects" the whole program in that way and fails to work unless the main program also runs POE. Just look at the moduels you yourself gave as examples: LWP::UserAgent::POE requires a blocking POE::Kernel->run, i.e. anybody who uses LWP::UserAgent::POE _has_ to use POE.

The same is true for any other module: A main program using EV cannot use POE (a main program using POE could use EV via POE, which is your - valid - point, but that completely besides the point :).

This is the crucial difference between "monopolising" frameworks such as POE/IO::Async/... and AnyEvent.

AnyEvent doesn't force your program to use AnyEvent. When a programmer uses a module that itself uses AnyEvent for event handling, that programmer can still run EV::loop, Gtk->main and so on. The programmer doesn't even need to know that the module he uses internally uses AnyEvent, it just works. That means that AnyEvent-using modules just work with POE, they just work under Gtk, they just work under EV and so on - the user doesn't need toc all a special AnyEvent->run method that blocks his program.

For module authors, this is all the difference in the world: as a module author, using POE means forcing all your users to use POE as well: Somebody *has* to call POE::Kernel->run for that module to work, and that someone has to be the main program. That means the usefulness of the module is limited to just that one event framework.

With AnyEvent, a module author can use events without forcing _his_ users to also use AnyEvent. AnyEvent-using modules work in POE programs just as fine as in Gtk programs, or Irssi.

POE-using modules can not ever work in Irssi (or urxvt), and don't coexist with e.g. IO::Async modules, as all these use the same logic: if you use Irssi, then Irssi runs the show, if you use EV, then EV runs the show, and if you use POE, then POE runs the show. If you use AnyEvent, then somebody else can run the show.

That's the crucial difference you keep failing to understand, rocco: _somebody_ has to call IO::Async::Loop->run, or POE::Kernel->run, or Gtk2->main, which will then run the rest of the show.

But nobody has to call AnyEvent->run (it doesn't even exist).

Before accusing others of spreading misinformation you have the moral right to inform yourself about what they actually say - I know it is hard to understand that AnyEvent indeed has a subtle but crucial difference to make it event-loop agnostic, given that almost every other event loop on the planet has the same "monopolising" structure as POE - but you still need to understand that AnyEvent is fundamentally difefrent here, and this diference is crucial for module authors.

"Multilooping with Coro"

As a concrete example of forcing multiple event loops to coexist in the same
process, using threads, I wrote this program a while ago:

http://data.plan9.de/multiloop

It manages to run a simple echo server coded using, respectively, IO::Async, POE, IO::Lambda, AnyEvent on top of Glib (or EV or any other event loop supported by AnyEvent), all at the same time in the same process. Getting POE to coexist with the others required patching the Perl interpreter at runtime, all others only needed minimal coaxing.

Note that this relates to my first post in this thread, not my previous one (using AnyEvent + one other monopolising event loop is of course trivial, this post is about the vastly more difficult problem of combining two or more of those _monopolising_ event loops, as normally, POE can't coexist with IO::Lambda for example, as they both want the main program to run only themselves).

Note also that not all of those event loops are neccessarily thread-safe, but most likely they are.

Note also that, yes, that means you can, in theory, use a POE module in an IO::Async program, but only by effectively creating "multiple programs" within the same process, using threads, in a rather inefficient way.

The throughput figures (sending data into the echo server and measuring the the return data bandwidth) gives a nice benchmark, too:

io::async 32.9mb/s
poe 14.3mb/s
io::lambda 9.3mb/s
stem 39.7mb/s
ae::handle 171.0mb/s

(take it with a grain of salt - I couldn't figure out how to use IO::Lambda in a truly nonblocking way as all its examples are using blocking I/O and it seems that making it truly nonblocking requires a whole load more code. Also, IO::Lambda::Server is line-buffered, which is partially responsible for it's low performance).

Marc:

Your argument is based upon the premise that POE cannot work beside other event loops. As an example, you say: "POE-using modules can not ever work in Irssi".

Here is a counter example. In 2007, Chris Williams demonstrated a POE-based IRC server running as an Irssi script. As part of his demonstration, he connected Irssi to the POE server running within the same client.

Irssi did not become a POE program.
POE did not "infect" Irssi.
POE did not "monopolise" Irssi's event loop.
POE::Kernel->run() wasn't called to run the IRC server.

If I seem to perpetually misunderstand the "crucial differences" you cite, it's because they contradict previously demonstrated truths.

As for POE::Kernel->run(), it is an event-loop agnostic way for applications to start the underlying main loop when they want. For example, POE::Loop::Gtk maps POE::Kernel->run() to Gtk->main() when using POE and Gtk together. As we've seen above, sometimes applications don't want to start the event loop, and that's okay.

Yes, we are obligated to inform ourselves about the things of which we speak, especially if we intend to speak the truth. We are also obligated to use civil language if we wish to participate in civil discourse.

Rocco... POE itself prevents it, as has been well-documented. When not calling the run method, POE will spill out warnings sooner or later, tellign the suer that the POE::Kernel->run code must be called after starting sessions (the infamous "Sessions were started, but POE::Kernel's run() method was never called to execute them.").

The fact that you can make hacks around POE by relying on internals and undocumented functionality has been demonstrated by me before in this thread (using Coro).

That doesn't mean anything to, and doesn't help, a module author, who wants that his/her module just works without resorting to hacks or forcing module _users_ to implement special hacks in their program. POE simply isn't transparent in that way, while AnyEvent is.

That's the crucial difference - my statement stands...

> Yes, we are obligated to inform ourselves about the things of which we speak

As documented by POE::Kernel itself, you are wrong. Maybe you should fix the POE module then? Do you really claim people have to find external hacks and distrust the actual documentation (and code!) that comes with POE that claims otherwise?

(I will not contribute further to this thread, others have pointed out the same, and if Rocco doesn't get it, it's his problem to keep)

My previous comment cites an example where POE runs alongside another event loop. The code would not have worked if your claims were true.

You dismiss this fact altogether too easily by mentioning a warning that POE emits when a user hasn't called POE::Kernel->run(). You claim that it says run() must be called after starting sessions. No, the warning doesn't specify when to call run().

In fact, the documentation explains that this "infamous" warning may be disabled by calling run() before starting sessions:

POE::Kernel will print a strong message if a program creates sessions but fails to call run(). If the lack of a run() call is deliberate, you can avoid the message by calling it before creating a session. run() at that point will return immediately, and POE::Kernel will be satisfied.

It's not POE's fault that you insist upon using "internals and undocumented functionality" to emulate the documented use of public methods.

I'm sorry that you don't trust POE. If there are specific issues you'd like addressed, please file bug reports, test cases, and/or patches. Bug trackers are much more reliable and expedient than rants about perceived defects in obscure places.

Suggestions and bugs against POE's documentation are also welcome. We're programmers, not technical writers, so documentation isn't our specialty. POE's documentation is about 188 printed pages (66 lines per page). The more "editors" we have looking it over, the better.

Thank you.