{"id":880,"date":"2022-08-19T06:38:31","date_gmt":"2022-08-19T06:38:31","guid":{"rendered":"https:\/\/craftcookcode.com\/?p=880"},"modified":"2024-07-10T02:37:54","modified_gmt":"2024-07-10T02:37:54","slug":"python-unpacking-a-comp-3-number","status":"publish","type":"post","link":"https:\/\/craftcookcode.com\/?p=880","title":{"rendered":"Python – Unpacking a COMP-3 number"},"content":{"rendered":"\n

New business requirement today.  We have some old mainframe files we have to run through Qlik Replicate.<\/p>\n\n\n\n

Three problems we have to overcome:<\/p>\n\n\n\n

    \n
  1. The files are in fixed width format<\/li>\n\n\n\n
  2. The files are in EBCDIC format; specifically Code 1047<\/a><\/li>\n\n\n\n
  3. Inside the records there are comp-3 packed fields\u00a0<\/li>\n<\/ol>\n\n\n\n

    The mainframe team did kindly provide us with a schema file that showed us the how many bytes make up each field so we could divide up the fixed width file by reading in a certain number of bytes per a field.<\/p>\n\n\n\n

    Python provided a decode function to decode the fields read to a readable format:<\/p>\n\n\n

    \nfocus_data_ascii = focus_data.decode("cp1047").rstrip()\n<\/pre><\/div>\n\n\n
    The hard part now was the comp-3 packed fields. They are made up with some bit magic and working with bits and shifts is not my strongest suite  <\/p>\n\n\n\n
    I have been a bit spoilt so far working with python and most problems can be solved by “find the module to do the magic for you.”<\/p>\n\n\n\n
    But after ages of scouring for a module to handle the conversion for me; I had a lot of false leads – testing questionable code with no luck.<\/p>\n\n\n\n
    Eventually I stumbles upon:<\/p>\n\n\n\n
    zorchenhimer\/cobol-packed-numbers.py<\/a><\/p>\n\n\n\n
    Thank goodness.<\/p>\n\n\n\n
    It still works with bits ‘n’ shift magic – but it works on the data that I have and now have readable text <\/p>\n\n\n\n
    I extended the code to fulfil the business requirements:<\/p>\n\n\n
    \n# Source https:\/\/gist.github.com\/zorchenhimer\/fd4d4208312d4175d106\ndef unpack_number(field, no_decimals):\n  \u00a0 """ Unpack a COMP-3 number. """\n  \u00a0 a = array('B', field)\n  \u00a0 value = float(0)\n\n\u00a0 \u00a0 # For all but last digit (half byte)\n  \u00a0 for focus_half_byte in a[:-1]:\n  \u00a0 \u00a0 \u00a0 value = (value * 100) + ( ( (focus_half_byte & 0xf0) >> 4) * 10) + (focus_half_byte & 0xf)\n\n\u00a0 \u00a0 # Last digit\n  \u00a0 focus_half_byte = a[-1]\n  \u00a0 value = (value * 10) + ((focus_half_byte & 0xf0) >> 4)\n\n\u00a0 \u00a0 # Negative\/Positve check. \u00a0If 0xd; it is a negative value\n  \u00a0 if (focus_half_byte & 0xf) == 0xd:\n  \u00a0 \u00a0 \u00a0 value = value * -1\n\n\u00a0 \u00a0 # If no_decimals = 0; it is just an int\n  \u00a0 if no_decimals == 0:\n  \u00a0 \u00a0 \u00a0 return_int = int(value)\n  \u00a0 \u00a0 \u00a0 return (return_int)\n  \u00a0 else:\n  \u00a0 \u00a0 \u00a0 return_float = value \/ pow(10, no_decimals)\n  \u00a0 \u00a0 \u00a0 return (return_float)\n<\/pre><\/div>","protected":false},"excerpt":{"rendered":"
    New business requirement today.  We have some old mainframe files we have to run through Qlik Replicate. Three problems we have to overcome: The mainframe team...<\/p>\n","protected":false},"author":1,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"om_disable_all_campaigns":false,"_monsterinsights_skip_tracking":false,"_monsterinsights_sitenote_active":false,"_monsterinsights_sitenote_note":"","_monsterinsights_sitenote_category":0,"footnotes":""},"categories":[13,18],"tags":[],"class_list":["post-880","post","type-post","status-publish","format-standard","hentry","category-code","category-python"],"aioseo_notices":[],"_links":{"self":[{"href":"https:\/\/craftcookcode.com\/index.php?rest_route=\/wp\/v2\/posts\/880","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/craftcookcode.com\/index.php?rest_route=\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/craftcookcode.com\/index.php?rest_route=\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/craftcookcode.com\/index.php?rest_route=\/wp\/v2\/users\/1"}],"replies":[{"embeddable":true,"href":"https:\/\/craftcookcode.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcomments&post=880"}],"version-history":[{"count":5,"href":"https:\/\/craftcookcode.com\/index.php?rest_route=\/wp\/v2\/posts\/880\/revisions"}],"predecessor-version":[{"id":1459,"href":"https:\/\/craftcookcode.com\/index.php?rest_route=\/wp\/v2\/posts\/880\/revisions\/1459"}],"wp:attachment":[{"href":"https:\/\/craftcookcode.com\/index.php?rest_route=%2Fwp%2Fv2%2Fmedia&parent=880"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/craftcookcode.com\/index.php?rest_route=%2Fwp%2Fv2%2Fcategories&post=880"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/craftcookcode.com\/index.php?rest_route=%2Fwp%2Fv2%2Ftags&post=880"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}