CURL – SFTP from PHP

Sample code to do a sftp from php

$ch = curl_init();

$localfile = ‘export-3.csv’;

$fp = fopen($localfile, ‘r’);

curl_setopt($ch, CURLOPT_URL, ‘sftp://user:pass@ftp.remote.com/’.$localfile);

curl_setopt($ch, CURLOPT_UPLOAD, 1);

curl_setopt($ch, CURLOPT_PROTOCOLS, CURLPROTO_SFTP);

curl_setopt($ch, CURLOPT_INFILE, $fp);

curl_setopt($ch, CURLOPT_INFILESIZE, filesize($localfile));

curl_exec ($ch);

$error_no = curl_errno($ch);

curl_close ($ch);

if ($error_no == 0) {

$error = ‘File uploaded succesfully.’;

} else {

$error = ‘File upload error.’;

}

 

FreeSwitch send receive Fax

Receiving fax in FreeSwitch is quite simple with mod_spandsp, but managing these faxes can be complex.

To help you, i decided to put here all the informations i have about my FreeSwitch configuration to receive faxes. Once you’ll be done with this tutorial, you sould have a nice system which will receive faxes and manage them easily.

STEP 1

Configuration of mod_spandsp
${CONF}/autoload_configs/spandsp.conf.xml You shouldn’t have to modify this file!
${CONF}/autoload_configs/fax.conf.xml

<configuration name="fax.conf" description="FAX application configuration">
    <settings>
    <param name="use-ecm"       value="true"/>
    <param name="verbose"       value="false"/> <!-- For debugging -->
    <param name="disable-v17"   value="false"/>
    <param name="ident"         value="My FAX IDENT"/> <!-- Fax IDENT -->
    <param name="header"        value="My FAX HEADER"/> <!-- Fax HEADER -->
    <param name="spool-dir"     value="/tmp"/> <!-- Répertoire de travail -->
    <param name="file-prefix"   value="faxrx"/> <!-- Sera ajouté au début du nom du fichier -->
    </settings>
</configuration>

Dialplan configuration

<extension name="inbound_t1_fax">
    <condition field="destination_number" expression="^0000$"> <!-- Change for your own DID -->
       	<action application="set" data="session_in_hangup_hook=true"/> <!-- Pass fax headers at hangup -->
        <action application="set" data="api_hangup_hook=system ${base_dir}/scripts/process-fax.sh '\\\${fax_remote_station_id}' '\\\${fax_local_station_id}' '\\\${fax_document_transferred_pages}' '\\\${fax_document_total_pages}' '\\\${fax_result_text}' '${caller_id_number}' '${uuid}'"/> <!-- Script who receive fax headers at hangup-->
       	<action application="python" data="process-rxfax"/> <!-- Fax reception management script -->
       	<action application="hangup"/>
    </condition>
</extension>

process-rxfax.py
Based on steffler.info

#
# Fax reception Python
# 2011-06-30
# David Laperle @ RSS
#

import sys                            # import the sys module for argv
import os                              # import the os modules
import time
import smtplib, email, datetime
import glob
import math
import string
from email.Encoders import encode_base64
from email.MIMEMultipart import MIMEMultipart
from email.MIMEAudio import MIMEAudio
from email.Header import Header
from email.MIMEText import MIMEText
from datetime import *
from freeswitch import *

## EDIT THESE VARIABLES TO MATCH YOUR NEEDS:

tiff2pscmd = '/usr/bin/tiff2ps'      # location of tiff2ps
tiff2pdfcmd = '/usr/bin/tiff2pdf'      # location of tiff2pdf
ncftpputcmd = '/usr/local/bin/ncftpput' # location of ncftpput
lprcmd = '/usr/bin/lpr'              # location of lpr
convertcmd = '/usr/bin/convert'      # location of convert
incomingfaxes = '/tmp/'              # change this to where your .tiffs are saved by mod_fax, trailing slash required
ftphost = 'host'
ftpuser = 'user'
ftppass = 'pass'
today = date.today().strftime("%Y%m%d")
tiff2psoptions = '-a -O '              # command line options go here 
logfile = "/usr/local/freeswitch/log/rxfax_" + today + ".log"

def sendErrorToMail(subject,reason):
    email_from = "from@domain.com"
    email_rcpt = "to@domain.com"
    msg = MIMEMultipart('alternative')
    msg.set_charset('utf-8')
    sub = Header(subject,'utf-8')
    msg['Subject'] = sub
    msg['From'] = email_from
    msg['To'] = email_rcpt
    textv = reason
    part1 = MIMEText(textv,'plain','utf-8')
    msg.attach(part1)
    mailer = smtplib.SMTP("server.smtp.com",25)
    mailer.sendmail(email_from, email_rcpt, msg.as_string())
    mailer.quit()

def writeToLog(session,msg):
    global logfile
    now = datetime.now().strftime("%Y-%m-%d %H:%M:%S")
    uuid = session.getVariable("uuid")
    towrite = now + " " + uuid + " " + msg + "\n"
    fh = open(logfile,"a")
    fh.write(towrite)
    fh.close()

def handler(session, args):

    #get required variables
    the_uuid = session.getVariable("uuid")
    the_recipient = session.getVariable("recipient")
    the_caller = session.getVariable("caller_id_number")
    the_dest = session.getVariable("destination_number")

    writeToLog(session,"Incoming Fax from " + the_caller)
    #test if FTP works before proceeding
    error = os.system(ncftpputcmd + " -m -u " + ftpuser + " -p " + ftppass + " " + ftphost + " / /usr/local/freeswitch/scripts/testfile.txt ")
    if error == 0:
        writeToLog(session,"Fax reception in progress ...")
        #answer the phone, save the fax file in .tiff format.    
        session.answer()
        session.execute("playback", "silence_stream://2000")
        session.execute("rxfax", incomingfaxes + "rxfax-" + the_uuid + ".tiff")
        writeToLog(session,"Fax reception done: " + incomingfaxes + "rxfax-" + the_uuid + ".tiff")
        # We split the TIFF so we can have different page size within the same fax
        writeToLog(session,"Splitting TIFF")
        os.system(convertcmd + " " + incomingfaxes + "rxfax-" + the_uuid + ".tiff " + incomingfaxes + "rxfax-" + the_uuid + "_%04d.tiff")
        # We send the TIFF to ps directly to the printer
        writeToLog(session,"Sending Fax to printer")
        pages = glob.glob(incomingfaxes + "rxfax-" + the_uuid + "_*.tiff")
        pages.sort()
        for page in pages:
            # For each page we want to be able to determine the height, but depending on the y-resolution, it can change a little, so we do a little math, to always put the height on the same resolution
            val = 0
            p = os.popen('/usr/bin/identify -format \"%h\" ' + page)
            height = p.readline()
            p.close()
            r = os.popen('/usr/bin/identify -format \"%y\" ' + page)
            resolution = r.readline()
            r.close()
            ardpi = string.split(resolution)
            dpi = int(ardpi[0])
            if dpi != 196:
                if dpi > 196:
                    val = (dpi/196)*int(height)
                else:
                    val = (196/dpi)*int(height)
            else:
                val = int(height)
            if int(val) > 2300:
                writeToLog(session,page + " is LEGAL!")
                os.system(tiff2pscmd + " -h14 " + page + " | " + lprcmd + " -P myprinter -o media=legal -o fitplot=true")
            else:
                writeToLog(session,page + " is LETTER!")
                os.system(tiff2pscmd + " -h11 " + page + " | " + lprcmd + " -P myprinter -o media=letter -o fitplot=true")
        writeToLog(session,"PDF creation in progress...")
        error = os.system(tiff2pdfcmd + " " + incomingfaxes + "rxfax-" + the_uuid + ".tiff -o " + incomingfaxes + "rxfax-" + the_uuid + ".pdf")
        if error == 0:
            # Generating JPG preview
            writeToLog(session,"Generating JPG preview file (continue on failure)...")
            os.system(convertcmd + " -scale 500 " + incomingfaxes + "rxfax-" + the_uuid + ".tiff[0] " + incomingfaxes + "rxfax-" + the_uuid + ".jpg")
            writeToLog(session,"Sending PDF to FTP server...")
            # We send the PDF on remote server
            error = os.system(ncftpputcmd + " -m -u " + ftpuser + " -p " + ftppass + " " + ftphost + " / " + incomingfaxes + "rxfax-" + the_uuid + ".pdf ")
            os.system(ncftpputcmd + " -m -u " + ftpuser + " -p " + ftppass + " " + ftphost + " / " + incomingfaxes + "rxfax-" + the_uuid + ".jpg ")
            if error == 0:
                # Everything went fine, party time!
                writeToLog(session, "Everything went fine, cleaning up tmp files...")
                os.system('rm -f ' + incomingfaxes + "rxfax-" + the_uuid + ".pdf " + incomingfaxes + "rxfax-" + the_uuid + ".tiff " + incomingfaxes + "rxfax-" + the_uuid + "_*.tiff " + incomingfaxes + "rxfax-" + the_uuid + ".jpg")
            else:
                # FTP failed, we're fucked!
                writeToLog(session,"FTP upload failed, hanging up")
                session.hangup()
        else:
            # PDF couldn't be created, we're fucked!
            #writeToLog(session,"PDF creation FAILED, hanging up")
            mysub = "Fax en provenance de " + the_caller + " à ÉCHOUÉ"
            mytxt = "Raison: Création du fichier PDF à échoué! Fichier à récupérer: " + incomingfaxes + "rxfax-" + the_uuid + ".tiff"
            writeToLog(session,mytxt)
            sendErrorToEmail(mysub,mytxt)
            session.hangup()
    else:
    # FTP host is not working, we cancel right now!
        mysub = "Fax en provenance de " + the_caller + " à ÉCHOUÉ"
        mytxt = "Raison: FTP sur " + ftphost + " est hors service! Erreur Fatal, impossible de récupérer!"
        writeToLog(session,mytxt)
        sendErrorToMail(mysub,mytxt)
        session.hangup()

process-fax.sh

# Réception des arguments
REMOTE_STATION=$1
LOCAL_STATION=$2
PAGES=$3
TOTAL_PAGES=$4
CODE=$5
CALLER=$6
UUID=$7

# On passe les variables à un script python qui fera le reste du travail
/usr/bin/python /usr/local/freeswitch/scripts/process-rxfax-mysql.py "$REMOTE_STATION" "$LOCAL_STATION" "$PAGES" "$TOTAL_PAGES" "$CODE" "$CALLER" "$UUID"

process-rxfax-mysql.py

import sys
import MySQLdb 

# Réception des paramètres
remote_station = sys.argv[1]
local_station = sys.argv[2]
pages = sys.argv[3]
total_pages = sys.argv[4]
code = sys.argv[5]
caller = sys.argv[6]
uuid = sys.argv[7]

#Variables MySQL
myHost = "mysqlhost"
myUser = "mysqluser"
myPass = "mysqlpass"
myDb = "mysqldb"

conn = MySQLdb.connect (host = myHost,
                           user = myUser,
                           passwd = myPass,
                           db = myDb)
cursor = conn.cursor()
cursor.execute("""
    INSERT INTO rss_faxes(uuid,caller_id,remote_station,local_station,pages,pages_total,code,date_received)
    VALUES
    (%s,%s,%s,%s,%s,%s,%s,NOW())
""", (uuid,caller,remote_station,local_station,pages,total_pages,code))

Structure de la DB MySQL

-- ----------------------------
-- Table structure for `rss_faxes`
-- ----------------------------
DROP TABLE IF EXISTS `rss_faxes`;
CREATE TABLE `rss_faxes` (
  `id_fax` INT(8) NOT NULL AUTO_INCREMENT,
  `uuid` VARCHAR(255) NOT NULL,
  `caller_id` VARCHAR(255) NOT NULL,
  `remote_station` VARCHAR(255) NOT NULL,
  `local_station` VARCHAR(255) NOT NULL,
  `pages` VARCHAR(3) NOT NULL,
  `pages_total` VARCHAR(3) NOT NULL,
  `code` VARCHAR(255) NOT NULL,
  `date_received` datetime NOT NULL,
  PRIMARY KEY  (`id_fax`)
) ENGINE=MyISAM AUTO_INCREMENT=95 DEFAULT CHARSET=latin1;

After that, all you have to do, is make a little interface to show these MySQL entries.

Multiple concurrent database connections in drupal 7

To configure a concurrent database where you are not required to do context switching use the following code snippet.

$target='default';
$key = 'db2';
$db2 = Database::getConnection($target,$key);
$result = $phpbb->query($sql,$args,$opts);

This assumes you have a database configured in your settings.php like the following:

$databases['db2']['default'] = array(
    'driver' => 'mysql',
    'database' => 'forum',
    'username' => 'username',
    'password' => 'password',
    'host' => 'mysql.host.com',
    'prefix' => 'db2_'
);

iOS5 HTTP Post with Json Object

NSArray *keys = [NSArray arrayWithObjects:@"longitude", @"latitude", nil];
NSArray *objects = [NSArray arrayWithObjects:longitude, latitude, nil];

NSDictionary *jsonDictionary = [NSDictionary dictionaryWithObjects:objects forKeys:keys];

if([NSJSONSerialization isValidJSONObject:jsonDictionary])
{
    __jsonData = [NSJSONSerialization dataWithJSONObject:jsonDictionary options:0 error:nil];
    __jsonString = [[NSString alloc]initWithData:__jsonData encoding:NSUTF8StringEncoding];
}

// Be sure to properly escape your url string.
NSMutableURLRequest *request = [NSMutableURLRequest requestWithURL:@"https://site.com...etc"];
[request setHTTPMethod:@"POST"];
[request setHTTPBody: __jsonData];
[request setValue:@"application/json" forHTTPHeaderField:@"Content-Type"];
[request setValue:[NSString stringWithFormat:@"%d", [__jsonData length]] forHTTPHeaderField:@"Content-Length"];  

NSError *errorReturned = nil;
NSURLResponse *theResponse =[[NSURLResponse alloc]init];
NSData *data = [NSURLConnection sendSynchronousRequest:request returningResponse:&theResponse error:&errorReturned];

if (errorReturned) {
    // Handle error.
}
else
{
    NSError *jsonParsingError = nil;
    NSArray *jsonArray = [NSJSONSerialization JSONObjectWithData:data options:NSJSONReadingMutableContainers|NSJSONReadingAllowFragments error:&jsonParsingError];
}

Apache Performance Tuning: KeepAlive to remove latency

TCP protocol closes the connection on every file request and a new request is required to be created for each file. Using KeepAlive we can push more than one file in same TCP Connection.

Apache is the most widely used web server on the Internet. Knowing how to get the most out of Apache is very important for a systems administrator. Optimizing Apache is always a balancing act. It’s a case of sacrificing one resource in order to obtain savings in another.

 

What is KeepAlive?

HTTP is a session less protocol. A connection is made to transfer a single file and closed once the transfer is complete. This keeps things simple but its not very efficient.

To improve efficiency something called KeepAlive was introduced. With KeepAlive the web browser and the web server agree to reuse the same connection to transfer multiple files.

Advantages of KeepAlive

  • Improves website speed: It reduces latency associated with HTTP transfers and delivers a better user experience.
  • Reduces CPU usage: On the server side enabling KeepAlive reduces CPU usage. Consider that a typical web page has dozens of different files such as images, stylesheets, javascript files etc. If KeepAlive is disabled a separate connection must be made for each of those files. Creating and closing connections has an overhead and doing it for every single file wastes CPU time.

Disadvantages of Keepalive

  • Increases memory usage: Enabling KeepAlive  increases memory usage on the server. Apache processes have to keep connections open waiting for new requests from established connections. While they are waiting they are occupying RAM that could be used to service other clients. If you turn off KeepAlive fewer apache processes will remain active. This will lower memory usage and allow Apache to serve more users.

When should you enable KeepAlive?

Deciding whether to enable KeepAlive or not depends on a number of different factors:

  • Server resources: How much RAM vs. how much CPU power you have? RAM is often the biggest limiting factor in a webserver. If you have little RAM you should turn off KeepAlive because having Apache processes hanging around while they wait for more requests from persistent connections is a waste of precious memory. If CPU power is limited then you want KeepAlive on because it reduces CPU load.
  • Types of sites: If you have pages with a lot of images or other files linked into them, KeepAlive will improve the user experience significantly. This is because a single connection will be used to transfer multiple files.
  • Traffic patterns: The type of traffic you get. If your web traffic is spread out evenly throughout a day then you should turn on KeepAlive. OTOH, if you have bursty traffic where a lot of concurrent users access your sites during a short time period KeepAlive will cause your RAM usage to skyrocket so you should turn it off.

Configure Apache KeepAlive settings

Open up apache’s configuration file and look for the following settings. On Centos this file is called httpd.conf and is located in /etc/httpd/conf. The following settings are noteworthy:

  • KeepAlive: Switches KeepAlive on or off. Put in “KeepAlive on” to turn it on and “KeepAlive off” to turn it off.
  • MaxKeepAliveRequests: The maximum number of requests a single persistent connection will service. A number between 50 and 75 would be plenty.
  • KeepAliveTimeout: How long should the server wait for new requests from connected clients. The default is 15 seconds which is way too high. Set it to between 1 and 5 seconds to avoid having processes wasting RAM while waiting for requests.
Other settings

KeepAlive affects other settings in your Apache configuration file even though they are not directly related. Here are the settings in an Apache prefork MPM webserver:

  • MaxClients: MaxClients is the maximum number of child processes launched by Apache to service incoming requests. With KeepAlive enabled you have will have a higher number of child processes active during peak times. So your MaxClients value may have to be increased.
  • MaxRequestsPerChild: The number of requests a child process will serve before it is killed and recreated. This is done to prevent memory leaks. When KeepAlive is turned on each persistent connection will count as one request. That effectively turns MaxRequestsPerChild into a maximum connections per child value. As a result you can set a lower MaxRequestsPerChild value if you allow KeepAlive. If you don’t allow KeepAlive you should increase the MaxRequestsPerChild value to prevent excessive CPU usage.

To set keep=alive parameters in Apache2  Httpd, you need to pay attention to 3 directives:

Keep Alive

KeepAliveTimeout

MaxKeepAliveRequests

KeepAlive Directive

Description: Enables HTTP persistent connections
Syntax: KeepAlive On|Off
Default: KeepAlive On
Context: server config, virtual host
Status: Core
Module: core

The Keep-Alive extension to HTTP/1.0 and the persistent connection feature of HTTP/1.1 provide long-lived HTTP sessions which allow multiple requests to be sent over the same TCP connection. In some cases this has been shown to result in an almost 50% speedup in latency times for HTML documents with many images. To enable Keep-Alive connections, set KeepAlive On.

For HTTP/1.0 clients, Keep-Alive connections will only be used if they are specifically requested by a client. In addition, a Keep-Alive connection with an HTTP/1.0 client can only be used when the length of the content is known in advance. This implies that dynamic content such as CGI output, SSI pages, and server-generated directory listings will generally not use Keep-Alive connections to HTTP/1.0 clients. For HTTP/1.1 clients, persistent connections are the default unless otherwise specified. If the client requests it, chunked encoding will be used in order to send content of unknown length over persistent connections.

When a client uses a Keep-Alive connection it will be counted as a single “request” for the MaxRequestsPerChild directive, regardless of how many requests are sent using the connection.

KeepAliveTimeout Directive

Description: Amount of time the server will wait for subsequent requests on a persistent connection
Syntax: KeepAliveTimeout seconds
Default: KeepAliveTimeout 5
Context: server config, virtual host
Status: Core
Module: core

The number of seconds Apache will wait for a subsequent request before closing the connection. Once a request has been received, the timeout value specified by the Timeout directive applies.

Setting KeepAliveTimeout to a high value may cause performance problems in heavily loaded servers. The higher the timeout, the more server processes will be kept occupied waiting on connections with idle clients.

In a name-based virtual host context, the value of the first defined virtual host (the default host) in a set of NameVirtualHost will be used. The other values will be ignored.

MaxKeepAliveRequests Directive

Description: Number of requests allowed on a persistent connection
Syntax: MaxKeepAliveRequests number
Default: MaxKeepAliveRequests 100
Context: server config, virtual host
Status: Core
Module: core

The MaxKeepAliveRequests directive limits the number of requests allowed per connection whenKeepAlive is on. If it is set to 0, unlimited requests will be allowed. We recommend that this setting be kept to a high value for maximum server performance.

For example:

<pre><code>MaxKeepAliveRequests 500</code></pre>