Zebra_cURL, a high performance PHP cURL library

Get the latest updates on this PHP library via RSS

Zebra_cURL is a high performance PHP library acting as a wrapper to PHP’s libcurl library, which not only allows the running of multiple requests at once asynchronously, in parallel, but also as soon as one thread finishes it can be processed right away without having to wait for the other threads in the queue to finish.

Also, each time a request is completed another one is added to the queue, thus keeping a constant number of threads running at all times and eliminating wasted CPU cycles from busy waiting. This result is a faster and more efficient way of processing large quantities of cURL requests (like fetching thousands of RSS feeds at once), drastically reducing processing time.

This script supports GET (with caching) and POST request, basic downloads as well as downloads from FTP servers, HTTP Authentication, and requests through proxy servers.

For maximum efficiency downloads are streamed (bytes downloaded are directly written to disk) removing the unnecessary strain from the server of having to read files into memory first, and then writing them to disk.

Zebra_cURL requires the PHP cURL extension to be enabled.

The code is heavily commented and generates no warnings/errors/notices when PHP’s error reporting level is set to E_ALL.

Top

Features review

  • supports GET (with caching) and POST request, basic downloads as well as downloads from FTP servers, HTTP Authentication, and requests through proxy servers
  • allows the running of multiple requests at once asynchronously, in parallel, but also as soon as one thread finishes it can be processed right away without having to wait for the other threads in the queue to finish
  • downloads are streamed (bytes downloaded are directly written to disk) removing the unnecessary strain from the server of having to read files into memory first, and then writing them to disk
  • provides a very detailed information about the made requests
  • has comprehensive documentation
  • code is heavily commented and generates no warnings/errors/notices when PHP’s error reporting level is set to E_ALL

Top

Requirements

PHP 5.0.2+ with the cURL extension installed

Top

Installation

Download the latest version, unpack it, and put it in a place accessible to your scripts.

Top

How to use

Fetch RSS feeds

<?php

function callback($result) {

    // remember, the "body" property of $result is run through
    // "htmlentities()", so you may need to "html_entity_decode" it

    // show everything
    print_r('<pre>');
    print_r($result->info);

}

require 'path/to/Zebra_cURL.php';

// instantiate the Zebra_cURL class
$curl = new Zebra_cURL();

// cache results 60 seconds
$curl->cache('cache', 60);

// get RSS feeds of some popular tech websites
$curl->get(array(
    'http://rss1.smashingmagazine.com/feed/',
    'http://allthingsd.com/feed/',
    'http://feeds.feedburner.com/nettuts',
    'http://www.webmonkey.com/feed/',
    'http://feeds.feedburner.com/alistapart/main',
), 'callback');

?>

Twitter Search

<?php

function callback($result) {

    // results from twitter is json-encoded;
    // remember, the "body" property of $result is run through
    // "htmlentities()" so we need to "html_entity_decode" it
    $result->body = json_decode(html_entity_decode($result->body));

    // show everything
    print_r('<pre>');
    print_r($result);

}

// include the library
require 'path/to/Zebra_cURL.php';

// instantiate the Zebra_cURL class
$curl = new Zebra_cURL();

// cache results 60 seconds
$curl->cache('cache', 60);

// search twitter for the "jquery" hashtag
$curl->get('http://search.twitter.com/search.json?q=' . urlencode('#jquery'), 'callback');

?>

Download an image

<?php

// include the library
require 'path/to/Zebra_cURL.php';

// instantiate the Zebra_cURL class
$curl = new Zebra_cURL();

// download one of the official twitter image
$curl->download('https://abs.twimg.com/a/1362101114/images/resources/twitter-bird-callout.png', 'cache');

?>

Top

Download

version 1.0.2
If you find this library to be useful to you, you can support the author by donating a small amount via PayPal:

Zebra_cURL is distributed under the LGPL.

In plain English, this means that you have the right to view and to modify the source code of this software, but if you modify and distribute it, you are required to license your copy under a LGPL-compatible license, and to make the entire source code of your derivation available to anybody you distribute the software to.

You also have the right to use this software together with software that has different licensing terms (including, but not limited to, commercial and closed-source software), and distribute the combined software, as long as state that your software contains portions licensed under the LGPL license, and provide information about where the LGPL licensed software can be downloaded.

If you distribute copies of this software you may not change the copyright or license of this software.


You may also like:

Top

Documentation

Documentation Become a ninja.
Read the comprehensive documentation.
Top

Changelog

Click on a version to expand/collapse information.

version 1.0.2 (August 29, 2013)
  • fixed a bug where the “type” argument of the “http_authentication” method could not be changed; thanks apmolsa;
  • fixed a bug where the “chmod” argument of the “cache” method could not be changed; thanks apmolsa;
  • fixed a bug where PHP’s htmlentities() function was accidentally being run on the response body of downloads;
  • the constructor now takes one argument specifying whether the response body should be run through PHP’s htmlentities() function;
version 1.0.1 (May 30, 2013)
  • fixed a bug where in PHP 5.2.7+ the library was triggering fatal error because I was using func_num_args() as an argument to another function;
  • the project is now also available on GitHub and as a package for Composer
version 1.0 (March 02, 2013)
  • initial release;

Top

25 responses to “Zebra_cURL, a high performance PHP cURL library”

Follow the comments via RSS
  • hassan, 2013-04-17, 14:45

    Hi;
    i want to show me Zebra-curl only $result->info but it show info and body. i put print_r(”); print_r($result->info); in callback function but it show body and info.
    why?
    Thanks.

    Reply
    • Stefan Gabos, 2013-04-17, 16:56

      i cannot confirm that…maybe you have multiple gets and multiple callback functions and one of them is displaying something and the other(s) something else

  • Nick, 2013-04-17, 15:14

    How to get contents of a webpage into a string, without automatically displaying webpage contents on calling $curl->get ?

    Reply
    • Stefan Gabos, 2013-04-17, 16:58

      the “get” method (or any other of the methods) is not displaying anything on the screen…the content (and more) is passed to a callback function. see the examples.

  • Nick, 2013-04-17, 17:54

    Hi, if I use your example (http://stefangabos.ro/wp-content/docs/Zebra_cURL/Zebra_cURL/Zebra_cURL.html#methodget) and then just delete

    print_r('<pre>'); 
    print_r($result); 

    in callback function it will still display the webpage content of fetched webpage.

    I’ve also removed

    $curl->cache('cache', 3600);

    but it still shows the webpage content simply on calling $curl->get…

    Reply
  • Stefan Gabos, 2013-04-17, 19:54

    it looks like you are setting the CURLOPT_RETURNTRANSFER option to 0/FALSE somewhere

    Reply
  • rooco, 2013-05-21, 10:05

    is this library support maintaining session for subsequent request?

    Reply
    • Stefan Gabos, 2013-05-22, 06:31

      what do you mean?

  • Jerome, 2013-06-02, 08:45

    Hi,

    I use zebra-curl to download an array of pdf files. However, the downloaded pdfs are output to browser (stdout). In the destination folder where the pdfs are supposed to be, are 0 byte pdf files with the correct names.

    Appreciate any help you can offer. Thanks!
    Jerome

    Reply
    • Stefan Gabos, 2013-06-02, 10:46

      please email me the code you’re using and, if possible, a link to one of those pdf, please. thank you!

    • Longhorn, 2013-11-29, 04:36

      I’m having the same problem, the downloaded PDF is being ouputed to the browser, so, how did you solved this issue? Thanks in advance

    • Longhorn, 2013-11-29, 05:20

      I’m using this code:

      $curl->download(‘http://www.server.com/thefile.pdf’, $this->tmpFile, array($this, ‘callbackCURL’));

      if it helps…

    • Stefan Gabos, 2013-11-29, 21:38

      i have no idea…I do have “output_buffering” turned on for PHP though and maybe that’s what’s making it work as expected for me

    • Longhorn, 2013-11-30, 00:33

      What is output_buffering and what it’s for? can you test it turning this off? May be that i’m calling callback inside a class? I think that using array($this, ‘callback’) is right… can you confirm this point?

  • Andy, 2013-06-07, 02:37

    I installed the script and I am getting “Fatal error: func_num_args(): Can’t be used as a function parameter in /home/username/mydomain.net/search/Zebra_cURL.php on line 1397″ using PHP Version 5.2.17. I fixed it by assigning it to a variable first, but it should be made safe for 5.2 very easily. Thanks.

    Reply
    • Stefan Gabos, 2013-06-07, 06:39

      thank you – it was reported before and I have it fixed it, but I just forgot to release it

  • sercankd, 2013-07-28, 16:08

    i use your class to maintain curl stuff, but i am unable to save login cookie to file, its 0 bytes.
    i use cookies function in post callback but it saves 0bytes file

    Reply
  • Hikaru, 2013-10-26, 15:13

    Hi, i want to download and save webpages from array of URLs (throught get() isn’t it?), then how do I naming the html file?

    e.g.
    array ( “url_1″, “url_2″) is passed to get() and I want to name the saved html file from “url_1″ to “A.html” and “url_2″ to “B.html”? Since its async, we dont know which one finished first and name the file by array’s order.

    Reply
  • Hikaru, 2013-10-26, 16:28

    Aahh…I see.. I figured out that info about the URL and curl is returned, my bad.
    Anyway thanks for great lib. :)

    Reply
  • Zaid Crowe, 2013-11-20, 17:42

    Hi :)

    So…great library, I’m trying to integrate it into a class im creating and have hit a bump!

    I post($this->url, $this->parameters, ‘tokenCallback’); from within a method of the class, PHP says that tokenCallback can’t be found (makes sense), so I defined tokenCallback as another method within my class. I then get an error stating that basically the $results variable isn’t being passed through to it.

    If I move tokenCallback outside of the class, into a global scope – everything works as expected, but this isn’t really gracefull as I’d like!

    How can I call my callback from within a class?

    
    curl = new Zebra_cURL();
    $this->curl->cache('cache', $this->cacheTime);//think about this for caching token, if we are rejected maybe thats the only time we re interogate api...
    $this->curl->option(CURLOPT_HTTPHEADER, $this->customHeaders);
    $this->curl->post($this->url, $this->parameters, 'tokenCallback');
    
    public function tokenCallback($result){
      if ($result->response[1] == CURLE_OK) {
        $body = str_replace(array('\\/','\"'),array('/','"'), html_entity_decode($result->body)); 
        $tokenStart = stripos($body,"");
        $rawtoken = substr($body, $tokenStart, $tokenEnd-$tokenStart);
        return $rawtoken;
      }else{
        return false;
      }
    }
    Reply
    • Longhorn, 2013-12-01, 13:56

      Hi Zaid, i’m not sure, but i think that you must call this way:

      $this->curl->post($this->url, $this->parameters, array($this,’tokenCallback’));

      if tokenCallback is within the same class

  • Nikola, 2014-02-04, 18:17

    Hi,

    Can you please tell me how to send multi thread post request via loop.

    When I use

    $curl = new Zebra_cURL();
    $curl->threads = 200;
    foreach($servers as $server) {
      $curl->post($server,$header, 'callback');
    }

    It is using just one single thread.

    Reply
    • Stefan Gabos, 2014-02-07, 10:25

      It should be

      $curl = new Zebra_cURL();
      $curl->threads = 200;
      $curl->post($servers, $header, 'callback');
      
  • Syed I.R, 2014-03-02, 21:22

    Hi,

    I’ve sent a PR on Github with a workaround for PHP Bug #61141.
    Without it, The CPU Usage hits 100%.

    Did several tests before i could figure the issue.

    Review and update please!

    Thanks.

    P.S Awesome script!

    Reply
  • Yaron, 2014-03-20, 16:29

    I tried to test your example of twitter feed from several sources and received the following error:
    Warning: curl_setopt_array() [function.curl-setopt-array]: open_basedir restriction in effect. File() is not within the allowed path(s): (/home/{myservername}/:/tmp:/var/tmp:/usr/local/lib/php/) in /home/{myservername}/domains/{myservername}.me/public_html/tests/Zebra-cURL/Zebra_cURL.php on line 1750

    DO you have any idea why I get this message?
    Thanks
    Blessings
    Yaron

    Reply

Leave a Reply

Your email address will not be published
You can use <strong>, <em>, <a>, <img>, <code>
Characters are not case-sensitive