PHP5-XML: Parse Large, Complex Files to mySQL (Proj 1 of Many)

This is the first of almost a dozen "mini-projects" we have in the pipeline. We are looking to create a long term relationship with a contract PHP expert. This is NOT a job for a new or inexperienced programmer. We are looking for a senior person who understands the PHP/XML/mySQL/Zend/LAMP technologies cold AND has expertise in architecting/implementing mission critical, enterprise-class solutions with PHP to create compelling user experiences. For this first piece, we are under particularly tight deadlines for this functionality, so time is of the essence! Before bid acceptance, let's be sure to talk (over Skype, etc.) to get to know each other a bit and start on the right footing/ If you do not meet these criteria, please do not waste either of our time. However, if you do, and are the right person, this could be the first in a set of lucrative, interesting, and rewarding engagements! ###### Quiz & Eval Results Auto Update via XML ##### _________________________________________________________________ We need to trigger, FTP, and parse an XML results file from one of our vendor's systems. The results include quiz and evaluation data from users which must then be inserted/updated/merged selectively into multiple tables of our mySQL database (schema defined and ready to go). Also perform a similar, but straight forward replace of the CME user table from a similarly generated demographics file (in .csv format). ## Deliverables ###### Requirements Flow ###### 1) automate the generation, downloading, and parsing of the XML file, using PHP to... a) do a scripted login to third party vendor's CME admin web page b) mirror a user's actions by submitting appropriate form data to the CME XML report generation page c) monitor the corresponding vendor FTP site and, once the requested XML file has been created, download it to our web server d) parse the XML file e) update the database with the results, being careful to do inserts vs. updates as business logic necessitates into various tables (biz rule definition and field mapping will be be provided once a coder is chosen and an NDA is executed) 2) repeat the above steps with the corresponding user demographics data (from an auto-generated .csv file, instead of an XML file). However, the biz logic of this is a simple replace of the whole "user" table (ie, no conditional updates needed) Note: the "full" historical XML file is approximately 37MB and thus will likely not support a DOM approach (ie, reading the entire file and executing XPath queries), rather a stream-based approach such as magicparser (for which we have a license) will probably be needed. However, once the historical file has been processed, the expectation is to run daily, incremental updates with the relatively small change files thereby reducing the performance impact. 3) setup cron job(s) to run automatically, passing in auto-incremented report date ranges to just get new info , logging runs and results 4) create an admin web interface page to allow manual triggering of the above conversion routine, based on the user's desired date ranges. * note: unfortunately, the code and XML design we get from the external vendor is poor, but nothing can be done about it As a result, "our" code must be written very robustly, with the ability to very thoroughly check for, handle, and document (via logs and emailed alerts) exceptions, out of bounds conditions, server availabilty, etc.. An ERD, full schema, sample PHP code (including the "magicparser" library), XML files, and remote server account information will be provided to the winning Developer, after NDA execution ###### Deliverables ###### 1) Complete and fully-functional working PHP source programs (un-encrypted by any source code control systems such as ioncube, etc.), mySQL .sql files (if any), and CRON Script (for RedHat Linux and OS X 10.4x) to manage automated processing 2) Fully commented code and/or developer documentation, all in English. 3) Any unit tests for the above. 4) Deliverables must be in ready-to-run condition Buyer's environment (PHP 5.21/mySQL 5.0.37/Apache 2.0.52). 5) Given the nature of this mini-project's access to 3rd party systems via form submissions and FTP, as well as parsing of quite large XML files, security and performance optimization is critical. Documentation detailing QA measures, security methods, and performance metrics is critical. 6) All deliverables will be considered "work made for hire" under U.S. Copyright law. Buyer will receive exclusive and complete copyrights to all work purchased. (No GPL, GNU, 3rd party components, etc. unless all copyright ramifications are explained AND AGREED TO by the buyer on the site per the coder's Seller Legal Agreement). ## Platform ###### Platform ###### Platform Server: LAMP/MAMP (PHP 5.21/mySQL 5.0.37/Apache 2.0.52) Browsers: Windows XP and Visat - IE 6.x, IE 7x and Firefox 1.5/2 OS X - Safari 2.x+ and Firefox 1.53+ Development: PHP coding/debugging via Zend Studio 5.5.0a/platofrm 3.02, a demo copy of which may be downloaded from: [login to view URL]