o SbR @sUdZddlZddlZddlZddlZddlZddlZddlmZm Z m Z m Z m Z m Z mZmZmZddlZddlmZddlmZmZmZmZddlmZddlmZddlmZdd lmZdd l m!Z!dd l"m#Z#dd l$m%Z%dd l&m'Z'ddl(m)Z)ddl*m+Z+m,Z,ddl-m.Z.ddl/m0Z0ej1ej2ej3ej4ej5ej6dZ7GdddZ8GdddZ9GdddeZ:e:e9j;dZdZ?e:e=d< e:e9j@dZAe:e=d< eAZBe:e=d< deded ed!eCfd"d#ZDd$eCded ed!efd%d&ZEeBfded'e:d!efd(d)ZFeBfd*e e eCefd'e:d!efd+d,ZGeBfd-e eCefd'e:d!efd.d/ZHd0ed!efd1d2ZId0ed'e:d!eeejJffd3d4ZKd5ed6eLd'e:d!eeejJffd7d8ZMd0ed'e:d!eeejJffd9d:ZNd0ed'e:d!eeejJffd;d<ZOd0ed'e:d!ejfd=d>ZPd0ed!e'fd?d@ZQd0ed!eCfdAdBZRd0ed!efdCdDZSd0ed!e)fdEdFZTd0ed!efdGdHZUd0ed!efdIdJZVd0ed!eLfdKdLZWd0ed!e!fdMdNZXd0ed!eYfdOdPZZd0ed!efdQdRZ[d0ed!e%fdSdTZ\d0ed!e#fdUdVZ]d5e^d6eLd'e:d!efdWdXZ_eBfded'e:d!efdYdZZ`dS)[a Tools for using Python's :mod:`json` module with BSON documents. This module provides two helper methods `dumps` and `loads` that wrap the native :mod:`json` methods and provide explicit BSON conversion to and from JSON. :class:`~bson.json_util.JSONOptions` provides a way to control how JSON is emitted and parsed, with the default being the Relaxed Extended JSON format. :mod:`~bson.json_util` can also generate Canonical or legacy `Extended JSON`_ when :const:`CANONICAL_JSON_OPTIONS` or :const:`LEGACY_JSON_OPTIONS` is provided, respectively. .. _Extended JSON: https://github.com/mongodb/specifications/blob/master/source/extended-json.rst Example usage (deserialization): .. doctest:: >>> from bson.json_util import loads >>> loads('[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$scope": {}, "$code": "function x() { return 1; }"}}, {"bin": {"$type": "80", "$binary": "AQIDBA=="}}]') [{'foo': [1, 2]}, {'bar': {'hello': 'world'}}, {'code': Code('function x() { return 1; }', {})}, {'bin': Binary(b'...', 128)}] Example usage with :const:`RELAXED_JSON_OPTIONS` (the default): .. doctest:: >>> from bson import Binary, Code >>> from bson.json_util import dumps >>> dumps([{'foo': [1, 2]}, ... {'bar': {'hello': 'world'}}, ... {'code': Code("function x() { return 1; }")}, ... {'bin': Binary(b"")}]) '[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$code": "function x() { return 1; }"}}, {"bin": {"$binary": {"base64": "AQIDBA==", "subType": "00"}}}]' Example usage (with :const:`CANONICAL_JSON_OPTIONS`): .. doctest:: >>> from bson import Binary, Code >>> from bson.json_util import dumps, CANONICAL_JSON_OPTIONS >>> dumps([{'foo': [1, 2]}, ... {'bar': {'hello': 'world'}}, ... {'code': Code("function x() { return 1; }")}, ... {'bin': Binary(b"")}], ... json_options=CANONICAL_JSON_OPTIONS) '[{"foo": [{"$numberInt": "1"}, {"$numberInt": "2"}]}, {"bar": {"hello": "world"}}, {"code": {"$code": "function x() { return 1; }"}}, {"bin": {"$binary": {"base64": "AQIDBA==", "subType": "00"}}}]' Example usage (with :const:`LEGACY_JSON_OPTIONS`): .. doctest:: >>> from bson import Binary, Code >>> from bson.json_util import dumps, LEGACY_JSON_OPTIONS >>> dumps([{'foo': [1, 2]}, ... {'bar': {'hello': 'world'}}, ... {'code': Code("function x() { return 1; }", {})}, ... {'bin': Binary(b"")}], ... json_options=LEGACY_JSON_OPTIONS) '[{"foo": [1, 2]}, {"bar": {"hello": "world"}}, {"code": {"$code": "function x() { return 1; }", "$scope": {}}}, {"bin": {"$binary": "AQIDBA==", "$type": "00"}}]' Alternatively, you can manually pass the `default` to :func:`json.dumps`. It won't handle :class:`~bson.binary.Binary` and :class:`~bson.code.Code` instances (as they are extended strings you can't provide custom defaults), but it will be faster as there is less recursion. .. note:: If your application does not need the flexibility offered by :class:`JSONOptions` and spends a large amount of time in the `json_util` module, look to `python-bsonjs `_ for a nice performance improvement. `python-bsonjs` is a fast BSON to MongoDB Extended JSON converter for Python built on top of `libbson `_. `python-bsonjs` works best with PyMongo when using :class:`~bson.raw_bson.RawBSONDocument`. N) AnyDictMappingOptionalSequenceTupleTypeUnioncast) EPOCH_AWARE)ALL_UUID_SUBTYPES UUID_SUBTYPEBinaryUuidRepresentation)Code) CodecOptions)DBRef) Decimal128)Int64)MaxKey)MinKey)ObjectId)Regex)RE_TYPESON) Timestamp)utc)ilmsuxc@eZdZdZ dZ dZdS)DatetimeRepresentationrN)__name__ __module__ __qualname__LEGACY NUMBERLONGISO8601r-r-5/tmp/pip-target-onvjaxws/lib/python/bson/json_util.pyr${s  r$c@r#)JSONModerr%r&N)r'r(r)r*RELAXED CANONICALr-r-r-r.r/s r/cseZdZUdZeed<eed<eed<eed<dddejfde dde ede ede edede d e d dffd d Z d e ffd d Zd ee e fffdd Zd e d dfddZZS) JSONOptionsa Encapsulates JSON options for :func:`dumps` and :func:`loads`. :Parameters: - `strict_number_long`: If ``True``, :class:`~bson.int64.Int64` objects are encoded to MongoDB Extended JSON's *Strict mode* type `NumberLong`, ie ``'{"$numberLong": "" }'``. Otherwise they will be encoded as an `int`. Defaults to ``False``. - `datetime_representation`: The representation to use when encoding instances of :class:`datetime.datetime`. Defaults to :const:`~DatetimeRepresentation.LEGACY`. - `strict_uuid`: If ``True``, :class:`uuid.UUID` object are encoded to MongoDB Extended JSON's *Strict mode* type `Binary`. Otherwise it will be encoded as ``'{"$uuid": "" }'``. Defaults to ``False``. - `json_mode`: The :class:`JSONMode` to use when encoding BSON types to Extended JSON. Defaults to :const:`~JSONMode.LEGACY`. - `document_class`: BSON documents returned by :func:`loads` will be decoded to an instance of this class. Must be a subclass of :class:`collections.MutableMapping`. Defaults to :class:`dict`. - `uuid_representation`: The :class:`~bson.binary.UuidRepresentation` to use when encoding and decoding instances of :class:`uuid.UUID`. Defaults to :const:`~bson.binary.UuidRepresentation.UNSPECIFIED`. - `tz_aware`: If ``True``, MongoDB Extended JSON's *Strict mode* type `Date` will be decoded to timezone aware instances of :class:`datetime.datetime`. Otherwise they will be naive. Defaults to ``False``. - `tzinfo`: A :class:`datetime.tzinfo` subclass that specifies the timezone from which :class:`~datetime.datetime` objects should be decoded. Defaults to :const:`~bson.tz_util.utc`. - `args`: arguments to :class:`~bson.codec_options.CodecOptions` - `kwargs`: arguments to :class:`~bson.codec_options.CodecOptions` .. seealso:: The specification for Relaxed and Canonical `Extended JSON`_. .. versionchanged:: 4.0 The default for `json_mode` was changed from :const:`JSONMode.LEGACY` to :const:`JSONMode.RELAXED`. The default for `uuid_representation` was changed from :const:`~bson.binary.UuidRepresentation.PYTHON_LEGACY` to :const:`~bson.binary.UuidRepresentation.UNSPECIFIED`. .. versionchanged:: 3.5 Accepts the optional parameter `json_mode`. .. versionchanged:: 4.0 Changed default value of `tz_aware` to False. json_modestrict_number_longdatetime_representation strict_uuidNclsargskwargsreturncs~|dd|d<|dr|dt|d<|tjtjtjdfvr#tdttt t|j |g|Ri|}|t jt j t j fvrCtd||_|jt j krq|rRtd|dtjfvr]td|dvretd d|_tj|_d |_|S|jt j kr|dvrtd |dtjfvrtd |dvrtd d |_tj|_d |_|Sd|_tj|_d|_|dur||_|dur||_|dur||_|S) Ntz_awareFtzinfoznJSONOptions.datetime_representation must be one of LEGACY, NUMBERLONG, or ISO8601 from DatetimeRepresentation.zQJSONOptions.json_mode must be one of LEGACY, RELAXED, or CANONICAL from JSONMode.z>> from bson.json_util import CANONICAL_JSON_OPTIONS >>> CANONICAL_JSON_OPTIONS.tz_aware True >>> json_options = CANONICAL_JSON_OPTIONS.with_options(tz_aware=False, tzinfo=None) >>> json_options.tz_aware False .. versionadded:: 3.12 rENr-)rFr=getattrrGr2)rAr9optsoptr-r-r. with_options]s   zJSONOptions.with_options)r'r(r)__doc__int__annotations__boolr/r0rrrr@strrDrrFrK __classcell__r-r-rBr.r2s< /C r2)r3LEGACY_JSON_OPTIONSCANONICAL_JSON_OPTIONSRELAXED_JSON_OPTIONSDEFAULT_JSON_OPTIONSobjr8r9r:cOs*|dt}tjt||g|Ri|S)aaHelper function that wraps :func:`json.dumps`. Recursive function that handles all BSON types including :class:`~bson.binary.Binary` and :class:`~bson.code.Code`. :Parameters: - `json_options`: A :class:`JSONOptions` instance used to modify the encoding of MongoDB Extended JSON types. Defaults to :const:`DEFAULT_JSON_OPTIONS`. .. versionchanged:: 4.0 Now outputs MongoDB Relaxed Extended JSON by default (using :const:`DEFAULT_JSON_OPTIONS`). .. versionchanged:: 3.4 Accepts optional parameter `json_options`. See :class:`JSONOptions`. json_options)poprUjsondumps _json_convert)rVr8r9rWr-r-r.rZs rZr cs4|dtfdd|d<tj|g|Ri|S)aHelper function that wraps :func:`json.loads`. Automatically passes the object_hook for BSON type conversion. Raises ``TypeError``, ``ValueError``, ``KeyError``, or :exc:`~bson.errors.InvalidId` on invalid MongoDB Extended JSON. :Parameters: - `json_options`: A :class:`JSONOptions` instance used to modify the decoding of MongoDB Extended JSON types. Defaults to :const:`DEFAULT_JSON_OPTIONS`. .. versionchanged:: 4.0 Now loads :class:`datetime.datetime` instances as naive by default. To load timezone aware instances utilize the `json_options` parameter. See :ref:`tz_aware_default_change` for an example. .. versionchanged:: 3.5 Parses Relaxed and Canonical Extended JSON as well as PyMongo's legacy format. Now raises ``TypeError`` or ``ValueError`` when parsing JSON type wrappers with values of the wrong type or any extra keys. .. versionchanged:: 3.4 Accepts optional parameter `json_options`. See :class:`JSONOptions`. rWcs t|SN)object_pairs_hook)pairsrWr-r.s zloads..r])rXrUrYloads)r r8r9r-r_r.ras rarWcsvt|drtfdd|DSt|dr)t|ttfs)tfdd|DSzt|WSty:|YSw)z]Recursive helper method that converts BSON types so they can be converted into json. itemsc3s"|] \}}|t|fVqdSr\r[).0kvr_r-r. s z _json_convert..__iter__c3s|]}t|VqdSr\rc)rdrfr_r-r.rgs) hasattrrrb isinstancerPbyteslistdefault TypeError)rVrWr-r_r.r[s   r[r^cCst|||Sr\) object_hookZdocument_class)r^rWr-r-r.r]sr]dctcCsrd|vrt|St|dtr$d|vr$t|dttdfr$t|Sd|vr-t||Sd|vr5t|Sd|vr=t|Sd|vrEt |Sd |vrWd |vrRt ||St ||Sd |vr_t |Sd |vrht ||Sd |vrndSd|vrvt|Sd|vr|d}t|d|dSd|vrt|Sd|vrt|Sd|vrt|Sd|vrt|Sd|vrt|Sd|vrt|S|S)N$oid$ref$id$db$date$regex$minKey$maxKey$binary$type$code$uuidz $undefined $numberLong $timestamptr$numberDecimal $dbPointer$regularExpression$symbol $numberInt $numberDouble)_parse_canonical_oidrjr=rPtype_parse_canonical_dbref_parse_canonical_datetime_parse_legacy_regex_parse_canonical_minkey_parse_canonical_maxkey_parse_legacy_binary_parse_canonical_binary_parse_canonical_code_parse_legacy_uuid_parse_canonical_int64r_parse_canonical_decimal128_parse_canonical_dbpointer_parse_canonical_regex_parse_canonical_symbol_parse_canonical_int32_parse_canonical_double)rprWZtspr-r-r.rosZ    rodoccCsJ|d}t|ttfs |Sd}|ddD] }|t|dO}qt||S)Nrvr$options)rjrPrkr= _RE_OPT_TABLEr)rpatternflagsrJr-r-r.rs rcCsdt|dkr td|ft|dtstd|f|jtjkr+tt |dSt |dS)z*Decode a JSON legacy $uuid to Python UUID.r%zBad $uuid, extra field(s): %sr|z$uuid must be a string: %s) lenrnrjrPuuid_representationr UNSPECIFIEDr from_uuiduuidUUID)rrWr-r-r.rs  rdatasubtypecCsn|tvr(|j}t||}|tjkr|S|tkrtj}n|tjkr#tj}||S|dkr2t t j |St||S)Nr) r rrrrr STANDARDZ PYTHON_LEGACYZas_uuidr rr)rrrWrZ binary_valuer-r-r._binary_or_uuid's      rcCsht|dtrd|d|d<t|dd}|dkr%t|dddd}t|d}t|||S)Nrz%02xlry)rjrMbase64 b64decodeencoder)rrWrrr-r-r.r<s rcCs|d}|d}|d}t|tstd|ft|tr#t|dkr*td|ft|dkr7td|ft|}t|t|d|S) NryrsubTypez#$binary base64 must be a string: %sr&z9$binary subType must be a string at most 2 characters: %sz?$binary must include only "base64" and "subType" components: %sr) rjrPrnrrrrrrM)rrWbinaryZb64rrr-r-r.rFs  rc Cs"|d}t|dkrtd|ft|tr |ddkr&|dd}d}nC|ddvr?|d d kr?|dd}|dd}n*|d dvrR|dd }|d d}n|d dvre|dd }|d d}n|}d }|d }d}|dkrtt||dd}|d|}tj|dj |t d}|r|dkrt|dkr|dd d \}} t|dt| d} n+t|dkrt|dddt|ddd} nt|dkrt|ddd} |ddkr| d9} |tj | d}|j r|jr||j}|S|j ddStt||S)z3Decode a JSON datetime to python datetime.datetime.rur%zBad $date, extra field(s): %sZNi)+-:r.ri@B%Y-%m-%dT%H:%M:%S) microsecondr<ri<r)secondsr<)rrnrjrPrfindrMfloatdatetimestrptimereplacersplit timedeltar;r< astimezonebsonZ_millis_to_datetime) rrWZdtmdtoffsetZ dot_indexrZawarehoursminutesZsecsr-r-r.rUsV              *    rcCs&t|dkr td|ft|dS)z1Decode a JSON ObjectId to bson.objectid.ObjectId.r%zBad $oid, extra field(s): %srq)rrnrrr-r-r.rs  rcC*|d}t|dkrtd|ft|S)z&Decode a JSON symbol to Python string.rr%zBad $symbol, extra field(s): %s)rrnrP)rsymbolr-r-r.r rcCs6|D] }|dvrtd|fqt|d|ddS)z%Decode a JSON code to bson.code.Code.)r{$scopezBad $code, extra field(s): %sr{r)scope)rnrr=)rkeyr-r-r.rs rcCsl|d}t|dkrtd|ft|dkrtd|f|d}t|ts/tdt|t|d|S) z(Decode a JSON regex to bson.regex.Regex.rr%z*Bad $regularExpression, extra field(s): %sr&zNBad $regularExpression must include only "pattern"and "options" components: %soptionszCBad $regularExpression options, options must be string, was type %sr)rrnrjrPrr)rregexrIr-r-r.rs    rcCs*t|d|dfd|ddi|S)z(Decode a JSON DBRef to bson.dbref.DBRef.rrrsdatabasertN)rrXrr-r-r.rs*rcCs|d}t|dkrtd|ft|trB|}|jdur&td|ft|jts3td|ft|dkr@td|f|Std |f) z9Decode a JSON (deprecated) DBPointer to bson.dbref.DBRef.rr%z"Bad $dbPointer, extra field(s): %sNz#Bad $dbPointer, extra field $db: %sz+Bad $dbPointer, $id must be an ObjectId: %sr&z+Bad $dbPointer, extra field(s) in DBRef: %sz$Bad $dbPointer, expected a DBRef: %s)rrnrjras_docridr)rZdbrefZ dbref_docr-r-r.rs     rcCB|d}t|dkrtd|ft|tstd|ft|S)z"Decode a JSON int32 to python int.rr%z"Bad $numberInt, extra field(s): %sz$numberInt must be string: %s)rrnrjrPrM)rZi_strr-r-r.r   rcCr)z(Decode a JSON int64 to bson.int64.Int64.r}r%z#Bad $numberLong, extra field(s): %s)rrnr)rZl_strr-r-r.rrrcCr)z%Decode a JSON double to python float.rr%z%Bad $numberDouble, extra field(s): %sz $numberDouble must be string: %s)rrnrjrPrrZd_strr-r-r.rrrcCr)z7Decode a JSON decimal128 to bson.decimal128.Decimal128.rr%z&Bad $numberDecimal, extra field(s): %sz!$numberDecimal must be string: %s)rrnrjrPrrr-r-r.rrrcCsJt|dtus|ddkrtd|ft|dkr"td|ftS)z,Decode a JSON MinKey to bson.min_key.MinKey.rwr%z$minKey value must be 1: %sBad $minKey, extra field(s): %s)rrMrnrrrr-r-r.rs  rcCsHt|dtus|ddkrtd|ft|dkr!td|ftS)z,Decode a JSON MaxKey to bson.max_key.MaxKey.rxr%z$maxKey value must be 1: %sr)rrMrnrrrr-r-r.rs   rcCsT|jtjkrtdt|fdd|fgSdtdt|fdd|fgiS)Nryrzrrr)r3r/r*rr b64encodedecode)rrrWr-r-r._encode_binary s ""rc Cst|tr dt|iSt|trt||dSt|tjr|jtj krl|j s4|j t d}|j dus4J|t krl|j |}|j|j|jfdkrKd}n|d}t|jd}|r^d|fnd }d d |d ||fiSt|}|jtjkr{d |iSd d t|iiS|jrt|trd t|iSt|ttfrd }|jtj@r|d7}|jtj@r|d7}|jtj @r|d7}|jtj!@r|d7}|jtj"@r|d7}|jtj#@r|d7}t|j$tr|j$}n|j$%d}|j&t'jkrt(d|fd|fgSdt(d|fd|fgiSt|t)r ddiSt|t*rddiSt|t+r+dt(d|j,fd|j-fgiSt|t.rM|j/dur=dt|iSt(dt|fd t|j/|fgSt|t0rZt1||j2|St|t3rft1|d!|St|t4j5r|j6rt0j7||j8d"}t1||j2|Sd#|j9iSt|t:rd$t|iSt|t;r|S|j&t'j?|rd(d)iSt>@|r|d!krd*nd+} d(| iS|j&t'jsJ,             $+(   "0 $  ;