Migrate Existing XML As Structured Content

Digest

Cascade Server's Web Services interface exposes to SOAP clients and interface for adding and editing assets. This allows repetitive or difficult tasks to be accomplished easily with any SOAP-compatible programming language. As a demonstration of these features, this guide will walk users through the task of importing many database entries from an old CMS (in the form of an XML dump) into Cascade server with PHP and SOAP calls. Web Services makes this process much quicker than a manual import.

Technical

Migrating Existing XML as Structured Content

Preparation for Migrating Existing XML as Structured Content:

To access Cascade Web Services with PHP, a working PHP installation (as well as an installation of Cascade Server) is necessary. To use the SOAP and Tidy interfaces in PHP, ensure that the following lines are in the php.ini file:

On Unix-like systems, ensure that the following lines are in the php.conf file:

The XML

The data that the import application will import is stored in an XML dump from a defunct CMS application. In this case, there are two types of data to be retrieved from the XML. Web pages consist of the metadata for a web page and a block of HTML content. Quotes are simply four fields which describe the quote: an id, a quote type, a quote source, and the body of the quote. The structure of the XML is:

<cms>  
.  
.  
.  
 <web_pages>  
<id>1</id>  
<title>Company Overview</title>  
<path>/company/index.html</path>  
<keywords>Our Company, Service providers, global business, e-economy solutions</keywords>  
<content><p>Within five years we have seen <i>global...</content>  
<metaDesc>A quick overview of our company</metaDesc>  
 </web_pages>  
 <web_pages>  
<id>2</id>  
<title>Mission Statement</title>  
<path>/company/mission.html</path>  
<keywords>Our Company, Service providers, global business, mission</keywords>  
<content><p>When our company began...</content>  
<metaDesc>The statement of our mission</metaDesc>  
 </web_pages>  

 <web_pages>  
<id>3</id>  
<title>ProductOne Overview</title>  
<path>/company/product1.html</path>  
<keywords>Our Company, Service providers, global business, ProductOne</keywords> 
<content><p>Advancements in research through...</content>  
<metaDesc>Feature Overview for ProductOne</metaDesc>  
 </web_pages>  
.  
.  
.  
 <quotes>  
<id>1</id>  
<type>Training</type>  
<source>Francois de La Rochefoucauld</source>  
<body>Good advice is something a man gives when...</body>  
 </quotes>  
  
 <quotes>  
    <id>2</id>  
    <type>Global</type>  
    <source>Jim Cantalupo</source>  
    <body>And ours is a business that requires discipline and focus.</body>  
 </quotes>  
.  
.  
.  
</cms>

Notice that the XML may contain other second-level tags, so a parser is created that is flexible enough to work with all of the data. The script will pull out the data for each web page and import them into an existing template in the system, while preserving the metadata and path. This will require a folder structure to hold the pages. Finally, the script will pull the fields of each quote from the XML and create a page using a data definition for quotes.

Implementation

( download code)

The scriptwill be segregated into two parts. The first part will parse the XML and construct two data structures, one containing all the web pages and another containing all the quotes. The second part will handle actually importing web pages and quotes into the system. The script will process HTML web page content with PHP's Tidy interface. Tidy will ensure that data is well-formed XHTML encoded in UTF-8, which is suitable for importing into Cascade. Finally, the PHP SOAP API will take the data as an associative array and make the appropriate Web Services calls to create page assets for each web page and quote.

Parsing the XML

PHP includes a built-in XML parser based on expat, which means that PHP XML Parsing functions can parse, but not validate XML. Ensure that the XML passed to PHP is valid. The parser will be an event-based parser, so some functions will be registered to run when the parser encounters certain XML elements (like character data or an opening tag). Keep track of the state of the XML between events. For these purposes, a simple stack will be sufficient (all that is necessary to know is if the current tag is part of quotes or webpages structure). Here is the code for setting up the parser:

Notice that some arrays have been initialized to hold all the web pages and quotes. The tag stack was also initialized, which will record the current tag structure. For instance, when looking at an id tag inside of aweb_pagestag, the stack will look like this:

This allows for looking at the current context of an event. Take a look at the event callbacks that were registered in the previous code.

/* This function will be called by the XML parser when it 
   * finds an opening tag */  
  function start_element($parser, $name, $attrs) {  
    global $tagstack;  
    global $pages;  
    global $quotes;  
    
    /* Push the current tag name onto our tag stack */  
    array_unshift($tagstack, $name);  
    
    /* If we have just entered the context of the web_pages table 
     * push a new web page data structure onto our $pages array */  
    if ($name == "web_pages")  
      array_push($pages, array());  
    
    /* If we have just entered the context of the quotes table 
     * push a new web page data structure onto our $quotes array */  
    if ($name == "quotes")  
      array_push($quotes, array());  
  }  
    
  /* This function will be called by the XML parser when it 
   * finds a closing tag */  
  function end_element($parser, $name) {  
    global $tagstack;  
    
    /* We are exiting a tag, so we want to pop it off of our 
     * tag stack */  
    array_shift($tagstack);  
  }

When the parser finds an opening tag, we want to put that tag onto the tagstack. If the opening tag is one the ones we are interested in (either quotes or web_pages), we should prepare the appropriate array by adding a new empty entry. When we encounter a closing tag, we should pop one tag off of the tag stack.

/* This function will be called by the XML parser when it 
   * finds character data between the opening and the closing 
   * of a tag */  
  function character_data($parser, $data) {  
    global $tagstack;  
    global $pages;  
    global $quotes;  
    
    /* If we aren't deep enough to be in "web_pages" or "quotes" 
     * don't do anything */  
    if (count($tagstack) &lt; 2)  
      return;  
    
    /* The tag we are currently looking at will be the first element 
     * on the tag stack. */  
    $current_tag = $tagstack[0];  
    
    /* If we are in the context of the web_pages table and 
     * are inside of a child tag (notice we check the second element 
     * of the stack */  
    if ($tagstack[1] == "web_pages") {  
      $last = count($pages) - 1;  
    
      /* If this is the first character data for this tag, then 
       * initialize the entry in this page's array */  
      if (!isset($pages[$last][$current_tag]))  
        $pages[$last][$current_tag] = "";  
    
      /* Append the character data to the current page's tag */  
      $pages[$last][$current_tag] .= $data;  
    }  
    
    /* If we are in the context of the web_pages table and 
     * are inside of a tag */  
    if ($tagstack[1] == "quotes") {  
      $last = count($quotes) - 1;      
      /* If this is the first character data for this tag, then 
       * initialize the entry in this quotes's array */  
      if (!isset($quotes[$last][$current_tag]))  
        $quotes[$last][$current_tag] = "";  
    
      /* Append the character data to the current page's tag */  
      $quotes[$last][$current_tag] .= $data;  
    }  
  }

When character data is found between tags, it is necessary to figure out what context it is in (by looking one level down on the tag stack) and record the data. Data should be placed in an associative array inside of $pages and $quotes. This way, a particular quote body will be $quotes[3]["body"], which will make it easier to access the data later. Notice that data is always appended to fields. This is because the XML parser handles all character content, including other XML that is parses from inside of contenttags of web_pages.

When the parser finishes its work, there should be two data structures, $pages and $quotes with the information from the XML file. Take a look at the functions that will process and import the data.

Initially, the import_web_pages() function will configure Tidy to output the type of XHTML needed. Next, it will initialize some arrays to hold paths. The paths will be an associative array that acts like a hash table to keep track of each unique path.

The script will now iterate over every page pulled from the XML file. Notice that it replaces the old HTML in the content field with tidied XHTML. We ensure that the content will be valid and well-formed XHTML after the wrapping it in Cascade's system-xml tag.

/* Here we iterate over each line of the table that this query returns. 
   * Each line will be an associative array which has keys equal to the 
   * field names and value equal to the field values */  
  foreach ($pages as $key =&gt; $page) {  
  
    /* Now we use the Tidy object to clean up the "content" field 
     * which contains the content for the main block of each page. 
     * We save this cleaned content back into the content field. */  
    $tidy-&gt;parseString($page["content"], $tidy_config, 'utf8');  
    $tidy-&gt;cleanRepair();  
    $page["content"] = tidy_get_output($tidy);  
  
    /* Cascade requires that the content we use be well-formed XML, 
     * thus we need to wrap the XML in a single root tag before 
     * delivering it to the CMS */  
    $line["content"] =  
         '&lt;system-xml xmlns:cascade="http://www.hannonhill.com/CascadeServer/"&gt;'  
         . $line["content"] . "&lt;/system-xml&gt;";  
  
    /* Finally we want to end the HTML tag to complete the markup */  
     $page["content"] . '&lt;/html&gt;';

Here we split the old CMS-style path field into a Cascade path and a system name for our new page. We know that path elements are separated by '/' so we look for the last one in the path. The string after that is the system name (once we remove the extension) and everything before it is the Cascade parent directory.

/* The old CMS had a field named "path" which stored the complete 
     * path of the page, including directory structure and filename. 
     * We want to separate this path out into these two parts, so 
     * that we can add the pages to Cascade */  
  
    /* We want to find the string length of the path */  
    $pathlen = strlen($page["path"]);  
  
    /* Next we find the position of the last (notice the use of strrpos) 
     * directory seperator (the old CMS used a "/"). */  
    $filename_loc = strrpos($page["path"], "/");  
  
    /* Now we grab the part of the string after this last "/" so we can use 
     * that as the page's system name */  
    $system_name = substr($page["path"],  
                          $filename_loc + 1,  
                          $pathlen - $filename_loc - 1);  
  
    /* Because Cascade does not use page extensions in the CMS, we 
     * want to remove the extension from the system name. In this case, 
     * the old CMS used the ".htm" extension */  
    $system_name = str_replace(".htm", "", $system_name);  
  
    /* We store the system name in the associative array, so that we 
     * can use it later. */  
    $page["__system_name"] = $system_name;  
      
    /* Now we want to transform the path into the part before the 
     * page name */  
    $page["path"] = substr($page["path"], 0, $filename_loc);

Finally, we store the path in our previously-initialized array and end the foreach loop.

The rest of import_web_pages() simply sorts the path array, creates a SOAP client and calls functions to add each folder ( add_folder()) and each web page ( add_page()). These functions will be described below.

Adding a Single Folder

The add_folder() function adds a single folder to Cascade. It takes in the path of the folder to add and removes the last path element as the folder name. Then it constructs an associative array that the SOAP API will turn into a SOAP message. When using SOAP be warned that even though your message might conform to the WSDL that you use, this does not mean that your message was a success. Be sure to check the response that the server returns with __getLastReponse() as shown below.

In PHP the associative array that the SOAP functions accept is structured in a 'tag name' => "tag value" fashion. To pass in multiple tags of a particular tag name, simply associate the 'tag name' key with array of values. This is demonstrated in theadd_page() function. Tags may be nested by associating particular tags with other associative arrays.

function add_folder($path, $client) {  
  /* We need to find the last element on the path, which will 
   * be the directory we want to create. We do this similarly to 
   * the way we separated the page name from the path */  
  
  /* Find the string length of the path */  
  $pathlen = strlen($path);  
  
  /* Find the position of the last path separator */  
  $folder_loc = strrpos($path, "/");  
  
  /* The folder name will be the substring after this separator */  
  $folder_name = substr($path,  
                        $folder_loc + 1,  
                        $pathlen - $folder_loc - 1);  
  
  /* The parent folder for our new folder will be the parent directory */  
  $path = substr($path, 0, $folder_loc + 1);  
  
  /* Now we fill an associative array that we will pass to the PHP 
   * SOAP functions. The structure of this array will be determined 
   * by the WSDL specification. PHP will interpret the structure 
   * and create appropriate XML to pass to the Web Services */  
  $create_params = array (  
   'authentication' =&gt; array(   
    'password' =&gt; "admin",  
    'username' =&gt; "admin"  
    ),  
  
   'asset' =&gt; array(  
    'folder' =&gt; array(  
      'name' =&gt; $folder_name,  
      'metadataSetPath' =&gt; "/Default",  
      'parentFolderPath' =&gt; $path)));  
  
  try {  
    /* Pass the create parameters through the PHP SOAP client. */  
    $response = $client-&gt;create($create_params);  
     
    /* If something goes wrong, it will be reported with  
     * __getLastResponse, so be sure to take a look at the  
     * message it returns. */  
    //print($client-&gt;__getLastResponse() . "\n");  
  
  } catch (Exception $e) {  
  
    /* The request will throw an exception when parameters do 
     * not conform to the WSDL. This will *not* catch errors 
     * in the parameters themselves */  
    print("Folder creation request failed to conform to the WSDL.\n");  
  
  }

Adding a Single Web Page

The add_page() function is very similar to the add_folder() function, although the request structure is more complex and we must ensure that the metadata we pass to Cascade is valid UTF-8 text.

/* This function adds a page to the Cascade server with web 
 * services. The page information should be stored in an 
 * associative array, $pdata. */  
function add_page($pdata, $client) {  
  
  /* Ensure that all the metadata is encoded in UTF-8 */  
  $pdata["title"] = utf8_encode($pdata["title"]);  
  $pdata["keywords"] = utf8_encode($pdata["keywords"]);  
  $pdata["metaDesc"] = utf8_encode($pdata["metaDesc"]);  
  
  /* Construct the parameters in the same way that we  
   * constructucted them for the folder, though creating 
   * a page requires a bit more information */  
  $create_params = array (  
   'authentication' =&gt; array(   
    'password' =&gt; "admin",  
    'username' =&gt; "admin"  
    ),  
  
   'asset' =&gt; array(  
    'page' =&gt; array(  
      'name' =&gt; $pdata["__system_name"],  
      'parentFolderPath' =&gt; $pdata["path"],  
      'metadataSetPath' =&gt; "/Default",  
      'configurationSetPath' =&gt; "/General/Simple Full HTML",  
      'xhtml' =&gt; $pdata["content"],  
      'metadata' =&gt; array(  
        'title' =&gt; $pdata["title"],  
        'keywords' =&gt; $pdata["keywords"],  
        'metaDescription' =&gt;$pdata["metaDesc"]))));  
  
  try {  
    $response = $client-&gt;create($create_params);  
  
    /* If something goes wrong, uncomment this line to read  
     * the server's response to your SOAP request */  
    //print($client-&gt;__getLastResponse());  
  
  } catch (Exception $e) {  
    print("Page creation request failed to conform to the WSDL.\n");  
  }  
  
}

Importing Quotes

Importing the quotes is very similar to importing the web pages with one major difference: instead of simply passing XHTML content to Cascade, we want to place values in a data definition. Before running this script, we created a data definition in Cascade named Quotewhich contained fields for the source, type, and body of each quote. Then we created a configuration set which would format this data definition. Here we see the data definition we used for each quote.

Next, you can see the code for importing the quotes. Notice the structure of the parameter array this time. Structured data definition nodes are packaged in an array inside of the structuredDataNodes tag. Each element is an entry of the structuredDataNode array. To pass more than one node, simply add another level of arrays inside of structuredDataNode.

Summary

The Web Service interface to Cascade Server eases automation of asset creation. PHP's built-in SOAP and XML support makes it useful for migrating to Cascade Server from another product. Content passed to Cascade may have to be converted to XHTML with PHP Tidy and to UTF-8 with the PHP function, utf8_encode(). The PHP SOAP API requires parameters be formed using associative arrays, whose keys describe the tag names. If the call succeeds the server's response can be accessed through the __getLastResponse method of the SoapClient object. Migration becomes far less tedious and time consuming with the use of Web Services.

Page Navigation

Related Links

Learning Levels

Announcements

Migrate Existing XML As Structured Content

Digest

Technical

Migrating Existing XML as Structured Content

Preparation for Migrating Existing XML as Structured Content:

The XML

Implementation

Parsing the XML

Adding a Single Folder

Adding a Single Web Page

Importing Quotes

Summary

Related Links

Page Navigation

Related Links

Learning Levels

Announcements

Migrate Existing XML As Structured Content

Digest

Technical

Migrating Existing XML as Structured Content

Preparation for Migrating Existing XML as Structured Content:

The XML

Implementation

Parsing the XML

Adding a Single Folder

Adding a Single Web Page

Importing Quotes

Summary

Related Links

Knowledge Base Beta Feedback Form