diff options
25 files changed, 7730 insertions, 0 deletions
diff --git a/Master/source/tlperl/Canary-Stability-2012.tar.gz b/Master/source/tlperl/Canary-Stability-2012.tar.gz Binary files differnew file mode 100644 index 00000000000..01d2c7209db --- /dev/null +++ b/Master/source/tlperl/Canary-Stability-2012.tar.gz diff --git a/Master/source/tlperl/JSON-2.94.tar.gz b/Master/source/tlperl/JSON-2.94.tar.gz Binary files differnew file mode 100644 index 00000000000..97e70802533 --- /dev/null +++ b/Master/source/tlperl/JSON-2.94.tar.gz diff --git a/Master/source/tlperl/JSON-XS-3.04.tar.gz b/Master/source/tlperl/JSON-XS-3.04.tar.gz Binary files differnew file mode 100644 index 00000000000..a2b56b2f69f --- /dev/null +++ b/Master/source/tlperl/JSON-XS-3.04.tar.gz diff --git a/Master/source/tlperl/Types-Serialiser-1.0.tar.gz b/Master/source/tlperl/Types-Serialiser-1.0.tar.gz Binary files differnew file mode 100644 index 00000000000..cf51068e1be --- /dev/null +++ b/Master/source/tlperl/Types-Serialiser-1.0.tar.gz diff --git a/Master/source/tlperl/common-sense-3.74.tar.gz b/Master/source/tlperl/common-sense-3.74.tar.gz Binary files differnew file mode 100644 index 00000000000..46b6f965167 --- /dev/null +++ b/Master/source/tlperl/common-sense-3.74.tar.gz diff --git a/Master/source/tlperl/tlperl.README b/Master/source/tlperl/tlperl.README index da4b332acd1..5dfce7c5a38 100644 --- a/Master/source/tlperl/tlperl.README +++ b/Master/source/tlperl/tlperl.README @@ -134,6 +134,17 @@ This module did not compile rightaway: Tk-DirSelect 1.12 +JSON support, added october 25 2017: + +JSON 2.94 +Canary::Stability 2012 +common::sense 3.74 +Types::Serialiser 1.0 +JSON::XS 3.04 + +I temporarily added pl2bat.[bin|pl] to tlperl/bin/ because dmake test +required it for one of these modules. + MODIFICATIONS FOR TEXLIVE - As explained above: our versions of lib/Config.pm and lib/Config_heavy.pl diff --git a/Master/tlpkg/tlperl/lib/perllocal.pod b/Master/tlpkg/tlperl/lib/perllocal.pod index 67d5c91c465..2520a5dcf31 100644 --- a/Master/tlpkg/tlperl/lib/perllocal.pod +++ b/Master/tlpkg/tlperl/lib/perllocal.pod @@ -768,3 +768,113 @@ C<EXE_FILES: > =back
+=head2 Tue Oct 24 21:29:33 2017: C<Module> L<JSON|JSON>
+
+=over 4
+
+=item *
+
+C<installed into: c:\tlperl\site\lib>
+
+=item *
+
+C<LINKTYPE: dynamic>
+
+=item *
+
+C<VERSION: 2.94>
+
+=item *
+
+C<EXE_FILES: >
+
+=back
+
+=head2 Tue Oct 24 21:38:19 2017: C<Module> L<Canary::Stability|Canary::Stability>
+
+=over 4
+
+=item *
+
+C<installed into: c:\tlperl\site\lib>
+
+=item *
+
+C<LINKTYPE: dynamic>
+
+=item *
+
+C<VERSION: 2012>
+
+=item *
+
+C<EXE_FILES: >
+
+=back
+
+=head2 Tue Oct 24 21:54:47 2017: C<Module> L<common::sense|common::sense>
+
+=over 4
+
+=item *
+
+C<installed into: c:\tlperl\site\lib>
+
+=item *
+
+C<LINKTYPE: dynamic>
+
+=item *
+
+C<VERSION: 3.74>
+
+=item *
+
+C<EXE_FILES: >
+
+=back
+
+=head2 Tue Oct 24 22:18:00 2017: C<Module> L<Types::Serialiser|Types::Serialiser>
+
+=over 4
+
+=item *
+
+C<installed into: c:\tlperl\site\lib>
+
+=item *
+
+C<LINKTYPE: dynamic>
+
+=item *
+
+C<VERSION: 1.0>
+
+=item *
+
+C<EXE_FILES: >
+
+=back
+
+=head2 Tue Oct 24 22:19:46 2017: C<Module> L<JSON::XS|JSON::XS>
+
+=over 4
+
+=item *
+
+C<installed into: c:\tlperl\site\lib>
+
+=item *
+
+C<LINKTYPE: dynamic>
+
+=item *
+
+C<VERSION: 3.04>
+
+=item *
+
+C<EXE_FILES: bin/json_xs>
+
+=back
+
diff --git a/Master/tlpkg/tlperl/site/lib/Canary/Stability.pm b/Master/tlpkg/tlperl/site/lib/Canary/Stability.pm new file mode 100644 index 00000000000..21d7c659736 --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/Canary/Stability.pm @@ -0,0 +1,207 @@ +=head1 NAME + +Canary::Stability - canary to check perl compatibility for schmorp's modules + +=head1 SYNOPSIS + + # in Makefile.PL + use Canary::Stability DISTNAME => 2001, MINIMUM_PERL_VERSION; + +=head1 DESCRIPTION + +This module is used by Schmorp's modules during configuration stage to +test the installed perl for compatibility with his modules. + +It's not, at this stage, meant as a tool for other module authors, +although in principle nothing prevents them from subscribing to the same +ideas. + +See the F<Makefile.PL> in L<Coro> or L<AnyEvent> for usage examples. + +=cut + +package Canary::Stability; + +BEGIN { + $VERSION = 2012; +} + +sub sgr { + # we just assume ANSI almost everywhere + # red 31, yellow 33, green 32 + local $| = 1; + + $ENV{PERL_CANARY_STABILITY_COLOUR} ne 0 + and ((-t STDOUT and length $ENV{TERM}) or $ENV{PERL_CANARY_STABILITY_COLOUR}) + and print "\e[$_[0]m"; +} + +sub import { + my (undef, $distname, $minvers, $minperl) = @_; + + $ENV{PERL_CANARY_STABILITY_DISABLE} + and return; + + $minperl ||= 5.008002; + + print <<EOF; + +*** +*** Canary::Stability COMPATIBILITY AND SUPPORT CHECK +*** ================================================= +*** +*** Hi! +*** +*** I do my best to provide predictable and reliable software. +*** +*** However, in recent releases, P5P (who maintain perl) have been +*** introducing regressions that are sometimes subtle and at other times +*** catastrophic, often for personal preferences with little or no concern +*** for existing code, most notably CPAN. +*** +*** For this reason, it has become very hard for me to maintain the level +*** of reliability and support I have committed myself to in the past, at +*** least with some perl versions: I simply can't keep up working around new +*** bugs or gratituous incompatibilities, and in turn you might suffer from +*** unanticipated problems. +*** +*** Therefore I have introduced a support and compatibility check, the results +*** of which follow below, together with a FAQ and some recommendations. +*** +*** This check is just to let you know that there might be a risk, so you can +*** make judgement calls on how to proceed - it will not keep the module from +*** installing or working. +*** +EOF + + if ($minvers > $VERSION) { + sgr 33; + print <<EOF; +*** The stability canary says: (nothing, it died of old age). +*** +*** Your Canary::Stability module (used by $distname) is too old. +*** This is not a fatal problem - while you might want to upgrade to version +*** $minvers (currently installed version: $VERSION) to get better support +*** status testing, you might also not want to care at all, and all will +*** be well as long $distname works well enough for you, as the stability +*** canary is only used when installing the distribution. +EOF + } elsif ($] < $minperl) { + + sgr 33; + print <<EOF; +*** The stability canary says: chirp (it seems concerned about something). +*** +*** Your perl version ($]) is older than the $distname distribution +*** likes ($minperl). This is not a fatal problem - the module might work +*** well with your version of perl, but it does mean the author likely +*** won't do anything to make it work if it breaks. +EOF + } elsif (defined $Internals::StabilityBranchVersion) { + # note to people studying this modules sources: + # the above test is not considered a clean or stable way to + # test for the stability branch. + + sgr 32; + print <<EOF; +*** The stability canary says: chirp! chirp! (it seems to be quite excited) +*** +*** It seems you are running schmorp's stability branch of perl. +*** All should be well, and if it isn't, you should report this as a bug +*** to the $distname author. +EOF + } elsif ($] < 5.021) { + #sgr 32; + print <<EOF; +*** The stability canary says: chirp! chirp! (it seems to be quite happy) +*** +*** Your version of perl ($]) is quite supported by $distname, nothing +*** else to be said, hope it comes in handy. +EOF + } else { + sgr 31; + print <<EOF; +*** The stability canary says: (nothing, it was driven away by harsh weather) +*** +*** It seems you are running perl version $], likely the "official" or +*** "standard" version. While there is nothing wrong with doing that, +*** standard perl versions 5.022 and up are not supported by $distname. +*** While this might be fatal, it might also be all right - if you run into +*** problems, you might want to downgrade your perl or switch to the +*** stability branch. +*** +*** If everything works fine, you can ignore this message. +EOF + sgr 0; + print <<EOF; +*** +*** Stability canary mini-FAQ: +*** +*** Do I need to do anything? +*** With luck, no. While some distributions are known to fail +*** already, most should probably work. This message is here +*** to alert you that your perl is not supported by $distname, +*** and if things go wrong, you either need to downgrade, or +*** sidegrade to the stability variant of your perl version, +*** or simply live with the consequences. +*** +*** What is this canary thing? +*** It's purpose is to check support status of $distname with +*** respect to your perl version. +*** +*** What is this "stability branch"? +*** It's a branch or fork of the official perl, by schmorp, to +*** improve stability and compatibility with existing modules. +*** +*** How can I skip this prompt on automated installs? +*** Set PERL_CANARY_STABILITY_NOPROMPT=1 in your environment. +*** More info is in the Canary::Stability manpage. +*** +*** Long version of this FAQ: http://stableperl.schmorp.de/faq.html +*** Stability Branch homepage: http://stableperl.schmorp.de/ +*** + +EOF + + unless ($ENV{PERL_CANARY_STABILITY_NOPROMPT}) { + require ExtUtils::MakeMaker; + + ExtUtils::MakeMaker::prompt ("Continue anyways? ", "y") =~ /^y/i + or die "FATAL: User aborted configuration of $distname.\n"; + } + } + + sgr 0; +} + +=head1 ENVIRONMENT VARIABLES + +=over 4 + +=item C<PERL_CANARY_STABILITY_NOPROMPT=1> + +Do not prompt the user on alert messages. + +=item C<PERL_CANARY_STABILITY_COLOUR=0> + +Disable use of colour. + +=item C<PERL_CANARY_STABILITY_COLOUR=1> + +Force use of colour. + +=item C<PERL_CANARY_STABILITY_DISABLE=1> + +Disable this modules functionality completely. + +=back + +=head1 AUTHOR + + Marc Lehmann <schmorp@schmorp.de> + http://software.schmorp.de/pkg/Canary-Stability.html + +=cut + +1 + diff --git a/Master/tlpkg/tlperl/site/lib/JSON.pm b/Master/tlpkg/tlperl/site/lib/JSON.pm new file mode 100644 index 00000000000..9170fa6925c --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/JSON.pm @@ -0,0 +1,1659 @@ +package JSON; + + +use strict; +use Carp (); +use Exporter; +BEGIN { @JSON::ISA = 'Exporter' } + +@JSON::EXPORT = qw(from_json to_json jsonToObj objToJson encode_json decode_json); + +BEGIN { + $JSON::VERSION = '2.94'; + $JSON::DEBUG = 0 unless (defined $JSON::DEBUG); + $JSON::DEBUG = $ENV{ PERL_JSON_DEBUG } if exists $ENV{ PERL_JSON_DEBUG }; +} + +my %RequiredVersion = ( + 'JSON::PP' => '2.27203', + 'JSON::XS' => '2.34', +); + +# XS and PP common methods + +my @PublicMethods = qw/ + ascii latin1 utf8 pretty indent space_before space_after relaxed canonical allow_nonref + allow_blessed convert_blessed filter_json_object filter_json_single_key_object + shrink max_depth max_size encode decode decode_prefix allow_unknown +/; + +my @Properties = qw/ + ascii latin1 utf8 indent space_before space_after relaxed canonical allow_nonref + allow_blessed convert_blessed shrink max_depth max_size allow_unknown +/; + +my @XSOnlyMethods = qw/allow_tags/; # Currently nothing + +my @PPOnlyMethods = qw/ + indent_length sort_by + allow_singlequote allow_bignum loose allow_barekey escape_slash as_nonblessed +/; # JSON::PP specific + + +# used in _load_xs and _load_pp ($INSTALL_ONLY is not used currently) +my $_INSTALL_DONT_DIE = 1; # When _load_xs fails to load XS, don't die. +my $_ALLOW_UNSUPPORTED = 0; +my $_UNIV_CONV_BLESSED = 0; + + +# Check the environment variable to decide worker module. + +unless ($JSON::Backend) { + $JSON::DEBUG and Carp::carp("Check used worker module..."); + + my $backend = exists $ENV{PERL_JSON_BACKEND} ? $ENV{PERL_JSON_BACKEND} : 1; + + if ($backend eq '1') { + $backend = 'JSON::XS,JSON::PP'; + } + elsif ($backend eq '0') { + $backend = 'JSON::PP'; + } + elsif ($backend eq '2') { + $backend = 'JSON::XS'; + } + $backend =~ s/\s+//g; + + my @backend_modules = split /,/, $backend; + while(my $module = shift @backend_modules) { + if ($module =~ /JSON::XS/) { + _load_xs($module, @backend_modules ? $_INSTALL_DONT_DIE : 0); + } + elsif ($module =~ /JSON::PP/) { + _load_pp($module); + } + elsif ($module =~ /JSON::backportPP/) { + _load_pp($module); + } + else { + Carp::croak "The value of environmental variable 'PERL_JSON_BACKEND' is invalid."; + } + last if $JSON::Backend; + } +} + + +sub import { + my $pkg = shift; + my @what_to_export; + my $no_export; + + for my $tag (@_) { + if ($tag eq '-support_by_pp') { + if (!$_ALLOW_UNSUPPORTED++) { + JSON::Backend::XS + ->support_by_pp(@PPOnlyMethods) if ($JSON::Backend->is_xs); + } + next; + } + elsif ($tag eq '-no_export') { + $no_export++, next; + } + elsif ( $tag eq '-convert_blessed_universally' ) { + my $org_encode = $JSON::Backend->can('encode'); + eval q| + require B; + local $^W; + no strict 'refs'; + *{"${JSON::Backend}\::encode"} = sub { + # only works with Perl 5.18+ + local *UNIVERSAL::TO_JSON = sub { + my $b_obj = B::svref_2object( $_[0] ); + return $b_obj->isa('B::HV') ? { %{ $_[0] } } + : $b_obj->isa('B::AV') ? [ @{ $_[0] } ] + : undef + ; + }; + $org_encode->(@_); + }; + | if ( !$_UNIV_CONV_BLESSED++ ); + next; + } + push @what_to_export, $tag; + } + + return if ($no_export); + + __PACKAGE__->export_to_level(1, $pkg, @what_to_export); +} + + +# OBSOLETED + +sub jsonToObj { + my $alternative = 'from_json'; + if (defined $_[0] and UNIVERSAL::isa($_[0], 'JSON')) { + shift @_; $alternative = 'decode'; + } + Carp::carp "'jsonToObj' will be obsoleted. Please use '$alternative' instead."; + return JSON::from_json(@_); +}; + +sub objToJson { + my $alternative = 'to_json'; + if (defined $_[0] and UNIVERSAL::isa($_[0], 'JSON')) { + shift @_; $alternative = 'encode'; + } + Carp::carp "'objToJson' will be obsoleted. Please use '$alternative' instead."; + JSON::to_json(@_); +}; + + +# INTERFACES + +sub to_json ($@) { + if ( + ref($_[0]) eq 'JSON' + or (@_ > 2 and $_[0] eq 'JSON') + ) { + Carp::croak "to_json should not be called as a method."; + } + my $json = JSON->new; + + if (@_ == 2 and ref $_[1] eq 'HASH') { + my $opt = $_[1]; + for my $method (keys %$opt) { + $json->$method( $opt->{$method} ); + } + } + + $json->encode($_[0]); +} + + +sub from_json ($@) { + if ( ref($_[0]) eq 'JSON' or $_[0] eq 'JSON' ) { + Carp::croak "from_json should not be called as a method."; + } + my $json = JSON->new; + + if (@_ == 2 and ref $_[1] eq 'HASH') { + my $opt = $_[1]; + for my $method (keys %$opt) { + $json->$method( $opt->{$method} ); + } + } + + return $json->decode( $_[0] ); +} + + + +sub true { $JSON::true } + +sub false { $JSON::false } + +sub null { undef; } + + +sub require_xs_version { $RequiredVersion{'JSON::XS'}; } + +sub backend { + my $proto = shift; + $JSON::Backend; +} + +#*module = *backend; + + +sub is_xs { + return $_[0]->backend->is_xs; +} + + +sub is_pp { + return $_[0]->backend->is_pp; +} + + +sub pureperl_only_methods { @PPOnlyMethods; } + + +sub property { + my ($self, $name, $value) = @_; + + if (@_ == 1) { + my %props; + for $name (@Properties) { + my $method = 'get_' . $name; + if ($name eq 'max_size') { + my $value = $self->$method(); + $props{$name} = $value == 1 ? 0 : $value; + next; + } + $props{$name} = $self->$method(); + } + return \%props; + } + elsif (@_ > 3) { + Carp::croak('property() can take only the option within 2 arguments.'); + } + elsif (@_ == 2) { + if ( my $method = $self->can('get_' . $name) ) { + if ($name eq 'max_size') { + my $value = $self->$method(); + return $value == 1 ? 0 : $value; + } + $self->$method(); + } + } + else { + $self->$name($value); + } + +} + + + +# INTERNAL + +sub __load_xs { + my ($module, $opt) = @_; + + $JSON::DEBUG and Carp::carp "Load $module."; + my $required_version = $RequiredVersion{$module} || ''; + + eval qq| + use $module $required_version (); + |; + + if ($@) { + if (defined $opt and $opt & $_INSTALL_DONT_DIE) { + $JSON::DEBUG and Carp::carp "Can't load $module...($@)"; + return 0; + } + Carp::croak $@; + } + $JSON::BackendModuleXS = $module; + return 1; +} + +sub _load_xs { + my ($module, $opt) = @_; + __load_xs($module, $opt) or return; + + my $data = join("", <DATA>); # this code is from Jcode 2.xx. + close(DATA); + eval $data; + JSON::Backend::XS->init($module); + + return 1; +}; + + +sub __load_pp { + my ($module, $opt) = @_; + + $JSON::DEBUG and Carp::carp "Load $module."; + my $required_version = $RequiredVersion{$module} || ''; + + eval qq| use $module $required_version () |; + + if ($@) { + if ( $module eq 'JSON::PP' ) { + $JSON::DEBUG and Carp::carp "Can't load $module ($@), so try to load JSON::backportPP"; + $module = 'JSON::backportPP'; + local $^W; # if PP installed but invalid version, backportPP redefines methods. + eval qq| require $module |; + } + Carp::croak $@ if $@; + } + $JSON::BackendModulePP = $module; + return 1; +} + +sub _load_pp { + my ($module, $opt) = @_; + __load_pp($module, $opt); + + JSON::Backend::PP->init($module); +}; + +# +# Helper classes for Backend Module (PP) +# + +package JSON::Backend::PP; + +sub init { + my ($class, $module) = @_; + + # name may vary, but the module should (always) be a JSON::PP + + local $^W; + no strict qw(refs); # this routine may be called after JSON::Backend::XS init was called. + *{"JSON::decode_json"} = \&{"JSON::PP::decode_json"}; + *{"JSON::encode_json"} = \&{"JSON::PP::encode_json"}; + *{"JSON::is_bool"} = \&{"JSON::PP::is_bool"}; + + $JSON::true = ${"JSON::PP::true"}; + $JSON::false = ${"JSON::PP::false"}; + + push @JSON::Backend::PP::ISA, 'JSON::PP'; + push @JSON::ISA, $class; + $JSON::Backend = $class; + $JSON::BackendModule = $module; + ${"$class\::VERSION"} = $module->VERSION; + + for my $method (@XSOnlyMethods) { + *{"JSON::$method"} = sub { + Carp::carp("$method is not supported in $module."); + $_[0]; + }; + } + + return 1; +} + +sub is_xs { 0 }; +sub is_pp { 1 }; + +# +# To save memory, the below lines are read only when XS backend is used. +# + +package JSON; + +1; +__DATA__ + + +# +# Helper classes for Backend Module (XS) +# + +package JSON::Backend::XS; + +sub init { + my ($class, $module) = @_; + + local $^W; + no strict qw(refs); + *{"JSON::decode_json"} = \&{"$module\::decode_json"}; + *{"JSON::encode_json"} = \&{"$module\::encode_json"}; + *{"JSON::is_bool"} = \&{"$module\::is_bool"}; + + $JSON::true = ${"$module\::true"}; + $JSON::false = ${"$module\::false"}; + + push @JSON::Backend::XS::ISA, $module; + push @JSON::ISA, $class; + $JSON::Backend = $class; + $JSON::BackendModule = $module; + ${"$class\::VERSION"} = $module->VERSION; + + if ( $module->VERSION < 3 ) { + eval 'package JSON::PP::Boolean'; + push @{"$module\::Boolean::ISA"}, qw(JSON::PP::Boolean); + } + + for my $method (@PPOnlyMethods) { + *{"JSON::$method"} = sub { + Carp::carp("$method is not supported in $module."); + $_[0]; + }; + } + + return 1; +} + +sub is_xs { 1 }; +sub is_pp { 0 }; + +sub support_by_pp { + my ($class, @methods) = @_; + + JSON::__load_pp('JSON::PP'); + + local $^W; + no strict qw(refs); + + for my $method (@methods) { + my $pp_method = JSON::PP->can($method) or next; + *{"JSON::$method"} = sub { + if (!$_[0]->isa('JSON::PP')) { + my $xs_self = $_[0]; + my $pp_self = JSON::PP->new; + for (@Properties) { + my $getter = "get_$_"; + $pp_self->$_($xs_self->$getter); + } + $_[0] = $pp_self; + } + $pp_method->(@_); + }; + } + + $JSON::DEBUG and Carp::carp("set -support_by_pp mode."); +} + +1; +__END__ + +=head1 NAME + +JSON - JSON (JavaScript Object Notation) encoder/decoder + +=head1 SYNOPSIS + + use JSON; # imports encode_json, decode_json, to_json and from_json. + + # simple and fast interfaces (expect/generate UTF-8) + + $utf8_encoded_json_text = encode_json $perl_hash_or_arrayref; + $perl_hash_or_arrayref = decode_json $utf8_encoded_json_text; + + # OO-interface + + $json = JSON->new->allow_nonref; + + $json_text = $json->encode( $perl_scalar ); + $perl_scalar = $json->decode( $json_text ); + + $pretty_printed = $json->pretty->encode( $perl_scalar ); # pretty-printing + +=head1 VERSION + + 2.93 + +=head1 DESCRIPTION + +This module is a thin wrapper for L<JSON::XS>-compatible modules with a few +additional features. All the backend modules convert a Perl data structure +to a JSON text as of RFC4627 (which we know is obsolete but we still stick +to; see below for an option to support part of RFC7159) and vice versa. +This module uses L<JSON::XS> by default, and when JSON::XS is not available, +this module falls back on L<JSON::PP>, which is in the Perl core since 5.14. +If JSON::PP is not available either, this module then falls back on +JSON::backportPP (which is actually JSON::PP in a different .pm file) +bundled in the same distribution as this module. You can also explicitly +specify to use L<Cpanel::JSON::XS>, a fork of JSON::XS by Reini Urban. + +All these backend modules have slight incompatibilities between them, +including extra features that other modules don't support, but as long as you +use only common features (most important ones are described below), migration +from backend to backend should be reasonably easy. For details, see each +backend module you use. + +=head1 CHOOSING BACKEND + +This module respects an environmental variable called C<PERL_JSON_BACKEND> +when it decides a backend module to use. If this environmental variable is +not set, it tries to load JSON::XS, and if JSON::XS is not available, it +falls back on JSON::PP, and then JSON::backportPP if JSON::PP is not available +either. + +If you always don't want it to fall back on pure perl modules, set the +variable like this (C<export> may be C<setenv>, C<set> and the likes, +depending on your environment): + + > export PERL_JSON_BACKEND=JSON::XS + +If you prefer Cpanel::JSON::XS to JSON::XS, then: + + > export PERL_JSON_BACKEND=Cpanel::JSON::XS,JSON::XS,JSON::PP + +You may also want to set this variable at the top of your test files, in order +not to be bothered with incompatibilities between backends (you need to wrap +this in C<BEGIN>, and set before actually C<use>-ing JSON module, as it decides +its backend as soon as it's loaded): + + BEGIN { $ENV{PERL_JSON_BACKEND}='JSON::backportPP'; } + use JSON; + +=head1 USING OPTIONAL FEATURES + +There are a few options you can set when you C<use> this module: + +=over + +=item -support_by_pp + + BEGIN { $ENV{PERL_JSON_BACKEND} = 'JSON::XS' } + + use JSON -support_by_pp; + + my $json = JSON->new; + # escape_slash is for JSON::PP only. + $json->allow_nonref->escape_slash->encode("/"); + +With this option, this module loads its pure perl backend along with +its XS backend (if available), and lets the XS backend to watch if you set +a flag only JSON::PP supports. When you do, the internal JSON::XS object +is replaced with a newly created JSON::PP object with the setting copied +from the XS object, so that you can use JSON::PP flags (and its slower +C<decode>/C<encode> methods) from then on. In other words, this is not +something that allows you to hook JSON::XS to change its behavior while +keeping its speed. JSON::XS and JSON::PP objects are quite different +(JSON::XS object is a blessed scalar reference, while JSON::PP object is +a blessed hash reference), and can't share their internals. + +To avoid needless overhead (by copying settings), you are advised not +to use this option and just to use JSON::PP explicitly when you need +JSON::PP features. + +=item -convert_blessed_universally + + use JSON -convert_blessed_universally; + + my $json = JSON->new->allow_nonref->convert_blessed; + my $object = bless {foo => 'bar'}, 'Foo'; + $json->encode($object); # => {"foo":"bar"} + +JSON::XS-compatible backend modules don't encode blessed objects by +default (except for their boolean values, which are typically blessed +JSON::PP::Boolean objects). If you need to encode a data structure +that may contain objects, you usually need to look into the structure +and replace objects with alternative non-blessed values, or enable +C<convert_blessed> and provide a C<TO_JSON> method for each object's +(base) class that may be found in the structure, in order to let the +methods replace the objects with whatever scalar values the methods +return. + +If you need to serialise data structures that may contain arbitrary +objects, it's probably better to use other serialisers (such as +L<Sereal> or L<Storable> for example), but if you do want to use +this module for that purpose, C<-convert_blessed_universally> option +may help, which tweaks C<encode> method of the backend to install +C<UNIVERSAL::TO_JSON> method (locally) before encoding, so that +all the objects that don't have their own C<TO_JSON> method can +fall back on the method in the C<UNIVERSAL> namespace. Note that you +still need to enable C<convert_blessed> flag to actually encode +objects in a data structure, and C<UNIVERSAL::TO_JSON> method +installed by this option only converts blessed hash/array references +into their unblessed clone (including private keys/values that are +not supposed to be exposed). Other blessed references will be +converted into null. + +This feature is experimental and may be removed in the future. + +=item -no_export + +When you don't want to import functional interfaces from a module, you +usually supply C<()> to its C<use> statement. + + use JSON (); # no functional interfaces + +If you don't want to import functional interfaces, but you also want to +use any of the above options, add C<-no_export> to the option list. + + # no functional interfaces, while JSON::PP support is enabled. + use JSON -support_by_pp, -no_export; + +=back + +=head1 FUNCTIONAL INTERFACE + +This section is taken from JSON::XS. C<encode_json> and C<decode_json> +are exported by default. + +This module also exports C<to_json> and C<from_json> for backward +compatibility. These are slower, and may expect/generate different stuff +from what C<encode_json> and C<decode_json> do, depending on their +options. It's better just to use Object-Oriented interfaces than using +these two functions. + +=head2 encode_json + + $json_text = encode_json $perl_scalar + +Converts the given Perl data structure to a UTF-8 encoded, binary string +(that is, the string contains octets only). Croaks on error. + +This function call is functionally identical to: + + $json_text = JSON->new->utf8->encode($perl_scalar) + +Except being faster. + +=head2 decode_json + + $perl_scalar = decode_json $json_text + +The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries +to parse that as an UTF-8 encoded JSON text, returning the resulting +reference. Croaks on error. + +This function call is functionally identical to: + + $perl_scalar = JSON->new->utf8->decode($json_text) + +Except being faster. + +=head2 to_json + + $json_text = to_json($perl_scalar[, $optional_hashref]) + +Converts the given Perl data structure to a Unicode string by default. +Croaks on error. + +Basically, this function call is functionally identical to: + + $json_text = JSON->new->encode($perl_scalar) + +Except being slower. + +You can pass an optional hash reference to modify its behavior, but +that may change what C<to_json> expects/generates (see +C<ENCODING/CODESET FLAG NOTES> for details). + + $json_text = to_json($perl_scalar, {utf8 => 1, pretty => 1}) + # => JSON->new->utf8(1)->pretty(1)->encode($perl_scalar) + +=head2 from_json + + $perl_scalar = from_json($json_text[, $optional_hashref]) + +The opposite of C<to_json>: expects a Unicode string and tries +to parse it, returning the resulting reference. Croaks on error. + +Basically, this function call is functionally identical to: + + $perl_scalar = JSON->new->decode($json_text) + +You can pass an optional hash reference to modify its behavior, but +that may change what C<from_json> expects/generates (see +C<ENCODING/CODESET FLAG NOTES> for details). + + $perl_scalar = from_json($json_text, {utf8 => 1}) + # => JSON->new->utf8(1)->decode($json_text) + +=head2 JSON::is_bool + + $is_boolean = JSON::is_bool($scalar) + +Returns true if the passed scalar represents either JSON::true or +JSON::false, two constants that act like C<1> and C<0> respectively +and are also used to represent JSON C<true> and C<false> in Perl strings. + +See L<MAPPING>, below, for more information on how JSON values are mapped to +Perl. + +=head1 COMMON OBJECT-ORIENTED INTERFACE + +This section is also taken from JSON::XS. + +The object oriented interface lets you configure your own encoding or +decoding style, within the limits of supported formats. + +=head2 new + + $json = JSON->new + +Creates a new JSON::XS-compatible backend object that can be used to de/encode JSON +strings. All boolean flags described below are by default I<disabled>. + +The mutators for flags all return the backend object again and thus calls can +be chained: + + my $json = JSON->new->utf8->space_after->encode({a => [1,2]}) + => {"a": [1, 2]} + +=head2 ascii + + $json = $json->ascii([$enable]) + + $enabled = $json->get_ascii + +If C<$enable> is true (or missing), then the C<encode> method will not +generate characters outside the code range C<0..127> (which is ASCII). Any +Unicode characters outside that range will be escaped using either a +single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence, +as per RFC4627. The resulting encoded JSON text can be treated as a native +Unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string, +or any other superset of ASCII. + +If C<$enable> is false, then the C<encode> method will not escape Unicode +characters unless required by the JSON syntax or other flags. This results +in a faster and more compact format. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this document. + +The main use for this flag is to produce JSON texts that can be +transmitted over a 7-bit channel, as the encoded JSON texts will not +contain any 8 bit characters. + + JSON->new->ascii(1)->encode([chr 0x10401]) + => ["\ud801\udc01"] + +=head2 latin1 + + $json = $json->latin1([$enable]) + + $enabled = $json->get_latin1 + +If C<$enable> is true (or missing), then the C<encode> method will encode +the resulting JSON text as latin1 (or iso-8859-1), escaping any characters +outside the code range C<0..255>. The resulting string can be treated as a +latin1-encoded JSON text or a native Unicode string. The C<decode> method +will not be affected in any way by this flag, as C<decode> by default +expects Unicode, which is a strict superset of latin1. + +If C<$enable> is false, then the C<encode> method will not escape Unicode +characters unless required by the JSON syntax or other flags. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this document. + +The main use for this flag is efficiently encoding binary data as JSON +text, as most octets will not be escaped, resulting in a smaller encoded +size. The disadvantage is that the resulting JSON text is encoded +in latin1 (and must correctly be treated as such when storing and +transferring), a rare encoding for JSON. It is therefore most useful when +you want to store data structures known to contain binary data efficiently +in files or databases, not when talking to other JSON encoders/decoders. + + JSON->new->latin1->encode (["\x{89}\x{abc}"] + => ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not) + +=head2 utf8 + + $json = $json->utf8([$enable]) + + $enabled = $json->get_utf8 + +If C<$enable> is true (or missing), then the C<encode> method will encode +the JSON result into UTF-8, as required by many protocols, while the +C<decode> method expects to be handled an UTF-8-encoded string. Please +note that UTF-8-encoded strings do not contain any characters outside the +range C<0..255>, they are thus useful for bytewise/binary I/O. In future +versions, enabling this option might enable autodetection of the UTF-16 +and UTF-32 encoding families, as described in RFC4627. + +If C<$enable> is false, then the C<encode> method will return the JSON +string as a (non-encoded) Unicode string, while C<decode> expects thus a +Unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs +to be done yourself, e.g. using the Encode module. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this document. + +Example, output UTF-16BE-encoded JSON: + + use Encode; + $jsontext = encode "UTF-16BE", JSON->new->encode ($object); + +Example, decode UTF-32LE-encoded JSON: + + use Encode; + $object = JSON->new->decode (decode "UTF-32LE", $jsontext); + +=head2 pretty + + $json = $json->pretty([$enable]) + +This enables (or disables) all of the C<indent>, C<space_before> and +C<space_after> (and in the future possibly more) flags in one call to +generate the most readable (or most compact) form possible. + +=head2 indent + + $json = $json->indent([$enable]) + + $enabled = $json->get_indent + +If C<$enable> is true (or missing), then the C<encode> method will use a multiline +format as output, putting every array member or object/hash key-value pair +into its own line, indenting them properly. + +If C<$enable> is false, no newlines or indenting will be produced, and the +resulting JSON text is guaranteed not to contain any C<newlines>. + +This setting has no effect when decoding JSON texts. + +=head2 space_before + + $json = $json->space_before([$enable]) + + $enabled = $json->get_space_before + +If C<$enable> is true (or missing), then the C<encode> method will add an extra +optional space before the C<:> separating keys from values in JSON objects. + +If C<$enable> is false, then the C<encode> method will not add any extra +space at those places. + +This setting has no effect when decoding JSON texts. You will also +most likely combine this setting with C<space_after>. + +Example, space_before enabled, space_after and indent disabled: + + {"key" :"value"} + +=head2 space_after + + $json = $json->space_after([$enable]) + + $enabled = $json->get_space_after + +If C<$enable> is true (or missing), then the C<encode> method will add an extra +optional space after the C<:> separating keys from values in JSON objects +and extra whitespace after the C<,> separating key-value pairs and array +members. + +If C<$enable> is false, then the C<encode> method will not add any extra +space at those places. + +This setting has no effect when decoding JSON texts. + +Example, space_before and indent disabled, space_after enabled: + + {"key": "value"} + +=head2 relaxed + + $json = $json->relaxed([$enable]) + + $enabled = $json->get_relaxed + +If C<$enable> is true (or missing), then C<decode> will accept some +extensions to normal JSON syntax (see below). C<encode> will not be +affected in anyway. I<Be aware that this option makes you accept invalid +JSON texts as if they were valid!>. I suggest only to use this option to +parse application-specific files written by humans (configuration files, +resource files etc.) + +If C<$enable> is false (the default), then C<decode> will only accept +valid JSON texts. + +Currently accepted extensions are: + +=over 4 + +=item * list items can have an end-comma + +JSON I<separates> array elements and key-value pairs with commas. This +can be annoying if you write JSON texts manually and want to be able to +quickly append elements, so this extension accepts comma at the end of +such items not just between them: + + [ + 1, + 2, <- this comma not normally allowed + ] + { + "k1": "v1", + "k2": "v2", <- this comma not normally allowed + } + +=item * shell-style '#'-comments + +Whenever JSON allows whitespace, shell-style comments are additionally +allowed. They are terminated by the first carriage-return or line-feed +character, after which more white-space and comments are allowed. + + [ + 1, # this comment not allowed in JSON + # neither this one... + ] + +=back + +=head2 canonical + + $json = $json->canonical([$enable]) + + $enabled = $json->get_canonical + +If C<$enable> is true (or missing), then the C<encode> method will output JSON objects +by sorting their keys. This is adding a comparatively high overhead. + +If C<$enable> is false, then the C<encode> method will output key-value +pairs in the order Perl stores them (which will likely change between runs +of the same script, and can change even within the same run from 5.18 +onwards). + +This option is useful if you want the same data structure to be encoded as +the same JSON text (given the same overall settings). If it is disabled, +the same hash might be encoded differently even if contains the same data, +as key-value pairs have no inherent ordering in Perl. + +This setting has no effect when decoding JSON texts. + +This setting has currently no effect on tied hashes. + +=head2 allow_nonref + + $json = $json->allow_nonref([$enable]) + + $enabled = $json->get_allow_nonref + +If C<$enable> is true (or missing), then the C<encode> method can convert a +non-reference into its corresponding string, number or null JSON value, +which is an extension to RFC4627. Likewise, C<decode> will accept those JSON +values instead of croaking. + +If C<$enable> is false, then the C<encode> method will croak if it isn't +passed an arrayref or hashref, as JSON texts must either be an object +or array. Likewise, C<decode> will croak if given something that is not a +JSON object or array. + +Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, +resulting in an invalid JSON text: + + JSON->new->allow_nonref->encode ("Hello, World!") + => "Hello, World!" + +=head2 allow_unknown + + $json = $json->allow_unknown ([$enable]) + + $enabled = $json->get_allow_unknown + +If C<$enable> is true (or missing), then C<encode> will I<not> throw an +exception when it encounters values it cannot represent in JSON (for +example, filehandles) but instead will encode a JSON C<null> value. Note +that blessed objects are not included here and are handled separately by +c<allow_nonref>. + +If C<$enable> is false (the default), then C<encode> will throw an +exception when it encounters anything it cannot encode as JSON. + +This option does not affect C<decode> in any way, and it is recommended to +leave it off unless you know your communications partner. + +=head2 allow_blessed + + $json = $json->allow_blessed([$enable]) + + $enabled = $json->get_allow_blessed + +See L<OBJECT SERIALISATION> for details. + +If C<$enable> is true (or missing), then the C<encode> method will not +barf when it encounters a blessed reference that it cannot convert +otherwise. Instead, a JSON C<null> value is encoded instead of the object. + +If C<$enable> is false (the default), then C<encode> will throw an +exception when it encounters a blessed object that it cannot convert +otherwise. + +This setting has no effect on C<decode>. + +=head2 convert_blessed + + $json = $json->convert_blessed([$enable]) + + $enabled = $json->get_convert_blessed + +See L<OBJECT SERIALISATION> for details. + +If C<$enable> is true (or missing), then C<encode>, upon encountering a +blessed object, will check for the availability of the C<TO_JSON> method +on the object's class. If found, it will be called in scalar context and +the resulting scalar will be encoded instead of the object. + +The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON> +returns other blessed objects, those will be handled in the same +way. C<TO_JSON> must take care of not causing an endless recursion cycle +(== crash) in this case. The name of C<TO_JSON> was chosen because other +methods called by the Perl core (== not by the user of the object) are +usually in upper case letters and to avoid collisions with any C<to_json> +function or method. + +If C<$enable> is false (the default), then C<encode> will not consider +this type of conversion. + +This setting has no effect on C<decode>. + +=head2 filter_json_object + + $json = $json->filter_json_object([$coderef]) + +When C<$coderef> is specified, it will be called from C<decode> each +time it decodes a JSON object. The only argument is a reference to the +newly-created hash. If the code references returns a single scalar (which +need not be a reference), this value (i.e. a copy of that scalar to avoid +aliasing) is inserted into the deserialised data structure. If it returns +an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the +original deserialised hash will be inserted. This setting can slow down +decoding considerably. + +When C<$coderef> is omitted or undefined, any existing callback will +be removed and C<decode> will not change the deserialised hash in any +way. + +Example, convert all JSON objects into the integer 5: + + my $js = JSON->new->filter_json_object (sub { 5 }); + # returns [5] + $js->decode ('[{}]'); # the given subroutine takes a hash reference. + # throw an exception because allow_nonref is not enabled + # so a lone 5 is not allowed. + $js->decode ('{"a":1, "b":2}'); + +=head2 filter_json_single_key_object + + $json = $json->filter_json_single_key_object($key [=> $coderef]) + +Works remotely similar to C<filter_json_object>, but is only called for +JSON objects having a single key named C<$key>. + +This C<$coderef> is called before the one specified via +C<filter_json_object>, if any. It gets passed the single value in the JSON +object. If it returns a single value, it will be inserted into the data +structure. If it returns nothing (not even C<undef> but the empty list), +the callback from C<filter_json_object> will be called next, as if no +single-key callback were specified. + +If C<$coderef> is omitted or undefined, the corresponding callback will be +disabled. There can only ever be one callback for a given key. + +As this callback gets called less often then the C<filter_json_object> +one, decoding speed will not usually suffer as much. Therefore, single-key +objects make excellent targets to serialise Perl objects into, especially +as single-key JSON objects are as close to the type-tagged value concept +as JSON gets (it's basically an ID/VALUE tuple). Of course, JSON does not +support this in any way, so you need to make sure your data never looks +like a serialised Perl hash. + +Typical names for the single object key are C<__class_whatever__>, or +C<$__dollars_are_rarely_used__$> or C<}ugly_brace_placement>, or even +things like C<__class_md5sum(classname)__>, to reduce the risk of clashing +with real hashes. + +Example, decode JSON objects of the form C<< { "__widget__" => <id> } >> +into the corresponding C<< $WIDGET{<id>} >> object: + + # return whatever is in $WIDGET{5}: + JSON + ->new + ->filter_json_single_key_object (__widget__ => sub { + $WIDGET{ $_[0] } + }) + ->decode ('{"__widget__": 5') + + # this can be used with a TO_JSON method in some "widget" class + # for serialisation to json: + sub WidgetBase::TO_JSON { + my ($self) = @_; + + unless ($self->{id}) { + $self->{id} = ..get..some..id..; + $WIDGET{$self->{id}} = $self; + } + + { __widget__ => $self->{id} } + } + +=head2 max_depth + + $json = $json->max_depth([$maximum_nesting_depth]) + + $max_depth = $json->get_max_depth + +Sets the maximum nesting level (default C<512>) accepted while encoding +or decoding. If a higher nesting level is detected in JSON text or a Perl +data structure, then the encoder and decoder will stop and croak at that +point. + +Nesting level is defined by number of hash- or arrayrefs that the encoder +needs to traverse to reach a given point or the number of C<{> or C<[> +characters without their matching closing parenthesis crossed to reach a +given character in a string. + +Setting the maximum depth to one disallows any nesting, so that ensures +that the object is only a single hash/object or array. + +If no argument is given, the highest possible setting will be used, which +is rarely useful. + +=head2 max_size + + $json = $json->max_size([$maximum_string_size]) + + $max_size = $json->get_max_size + +Set the maximum length a JSON text may have (in bytes) where decoding is +being attempted. The default is C<0>, meaning no limit. When C<decode> +is called on a string that is longer then this many bytes, it will not +attempt to decode the string but throw an exception. This setting has no +effect on C<encode> (yet). + +If no argument is given, the limit check will be deactivated (same as when +C<0> is specified). + +=head2 encode + + $json_text = $json->encode($perl_scalar) + +Converts the given Perl value or data structure to its JSON +representation. Croaks on error. + +=head2 decode + + $perl_scalar = $json->decode($json_text) + +The opposite of C<encode>: expects a JSON text and tries to parse it, +returning the resulting simple scalar or reference. Croaks on error. + +=head2 decode_prefix + + ($perl_scalar, $characters) = $json->decode_prefix($json_text) + +This works like the C<decode> method, but instead of raising an exception +when there is trailing garbage after the first JSON object, it will +silently stop parsing there and return the number of characters consumed +so far. + +This is useful if your JSON texts are not delimited by an outer protocol +and you need to know where the JSON text ends. + + JSON->new->decode_prefix ("[1] the tail") + => ([1], 3) + +=head1 ADDITIONAL METHODS + +The following methods are for this module only. + +=head2 backend + + $backend = $json->backend + +Since 2.92, C<backend> method returns an abstract backend module used currently, +which should be JSON::Backend::XS (which inherits JSON::XS or Cpanel::JSON::XS), +or JSON::Backend::PP (which inherits JSON::PP), not to monkey-patch the actual +backend module globally. + +If you need to know what is used actually, use C<isa>, instead of string comparison. + +=head2 is_xs + + $boolean = $json->is_xs + +Returns true if the backend inherits JSON::XS or Cpanel::JSON::XS. + +=head2 is_pp + + $boolean = $json->is_pp + +Returns true if the backend inherits JSON::PP. + +=head2 property + + $settings = $json->property() + +Returns a reference to a hash that holds all the common flag settings. + + $json = $json->property('utf8' => 1) + $value = $json->property('utf8') # 1 + +You can use this to get/set a value of a particular flag. + +=head1 INCREMENTAL PARSING + +This section is also taken from JSON::XS. + +In some cases, there is the need for incremental parsing of JSON +texts. While this module always has to keep both JSON text and resulting +Perl data structure in memory at one time, it does allow you to parse a +JSON stream incrementally. It does so by accumulating text until it has +a full JSON object, which it then can decode. This process is similar to +using C<decode_prefix> to see if a full JSON object is available, but +is much more efficient (and can be implemented with a minimum of method +calls). + +This module will only attempt to parse the JSON text once it is sure it +has enough text to get a decisive result, using a very simple but +truly incremental parser. This means that it sometimes won't stop as +early as the full parser, for example, it doesn't detect mismatched +parentheses. The only thing it guarantees is that it starts decoding as +soon as a syntactically valid JSON text has been seen. This means you need +to set resource limits (e.g. C<max_size>) to ensure the parser will stop +parsing in the presence if syntax errors. + +The following methods implement this incremental parser. + +=head2 incr_parse + + $json->incr_parse( [$string] ) # void context + + $obj_or_undef = $json->incr_parse( [$string] ) # scalar context + + @obj_or_empty = $json->incr_parse( [$string] ) # list context + +This is the central parsing function. It can both append new text and +extract objects from the stream accumulated so far (both of these +functions are optional). + +If C<$string> is given, then this string is appended to the already +existing JSON fragment stored in the C<$json> object. + +After that, if the function is called in void context, it will simply +return without doing anything further. This can be used to add more text +in as many chunks as you want. + +If the method is called in scalar context, then it will try to extract +exactly I<one> JSON object. If that is successful, it will return this +object, otherwise it will return C<undef>. If there is a parse error, +this method will croak just as C<decode> would do (one can then use +C<incr_skip> to skip the erroneous part). This is the most common way of +using the method. + +And finally, in list context, it will try to extract as many objects +from the stream as it can find and return them, or the empty list +otherwise. For this to work, there must be no separators (other than +whitespace) between the JSON objects or arrays, instead they must be +concatenated back-to-back. If an error occurs, an exception will be +raised as in the scalar context case. Note that in this case, any +previously-parsed JSON texts will be lost. + +Example: Parse some JSON arrays/objects in a given string and return +them. + + my @objs = JSON->new->incr_parse ("[5][7][1,2]"); + +=head2 incr_text + + $lvalue_string = $json->incr_text + +This method returns the currently stored JSON fragment as an lvalue, that +is, you can manipulate it. This I<only> works when a preceding call to +C<incr_parse> in I<scalar context> successfully returned an object. Under +all other circumstances you must not call this function (I mean it. +although in simple tests it might actually work, it I<will> fail under +real world conditions). As a special exception, you can also call this +method before having parsed anything. + +That means you can only use this function to look at or manipulate text +before or after complete JSON objects, not while the parser is in the +middle of parsing a JSON object. + +This function is useful in two cases: a) finding the trailing text after a +JSON object or b) parsing multiple JSON objects separated by non-JSON text +(such as commas). + +=head2 incr_skip + + $json->incr_skip + +This will reset the state of the incremental parser and will remove +the parsed text from the input buffer so far. This is useful after +C<incr_parse> died, in which case the input buffer and incremental parser +state is left unchanged, to skip the text parsed so far and to reset the +parse state. + +The difference to C<incr_reset> is that only text until the parse error +occurred is removed. + +=head2 incr_reset + + $json->incr_reset + +This completely resets the incremental parser, that is, after this call, +it will be as if the parser had never parsed anything. + +This is useful if you want to repeatedly parse JSON objects and want to +ignore any trailing data, which means you have to reset the parser after +each successful decode. + +=head1 MAPPING + +Most of this section is also taken from JSON::XS. + +This section describes how the backend modules map Perl values to JSON values and +vice versa. These mappings are designed to "do the right thing" in most +circumstances automatically, preserving round-tripping characteristics +(what you put in comes out as something equivalent). + +For the more enlightened: note that in the following descriptions, +lowercase I<perl> refers to the Perl interpreter, while uppercase I<Perl> +refers to the abstract Perl language itself. + +=head2 JSON -> PERL + +=over 4 + +=item object + +A JSON object becomes a reference to a hash in Perl. No ordering of object +keys is preserved (JSON does not preserver object key ordering itself). + +=item array + +A JSON array becomes a reference to an array in Perl. + +=item string + +A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON +are represented by the same codepoints in the Perl string, so no manual +decoding is necessary. + +=item number + +A JSON number becomes either an integer, numeric (floating point) or +string scalar in perl, depending on its range and any fractional parts. On +the Perl level, there is no difference between those as Perl handles all +the conversion details, but an integer may take slightly less memory and +might represent more values exactly than floating point numbers. + +If the number consists of digits only, this module will try to represent +it as an integer value. If that fails, it will try to represent it as +a numeric (floating point) value if that is possible without loss of +precision. Otherwise it will preserve the number as a string value (in +which case you lose roundtripping ability, as the JSON number will be +re-encoded to a JSON string). + +Numbers containing a fractional or exponential part will always be +represented as numeric (floating point) values, possibly at a loss of +precision (in which case you might lose perfect roundtripping ability, but +the JSON number will still be re-encoded as a JSON number). + +Note that precision is not accuracy - binary floating point values cannot +represent most decimal fractions exactly, and when converting from and to +floating point, this module only guarantees precision up to but not including +the least significant bit. + +=item true, false + +These JSON atoms become C<JSON::true> and C<JSON::false>, +respectively. They are overloaded to act almost exactly like the numbers +C<1> and C<0>. You can check whether a scalar is a JSON boolean by using +the C<JSON::is_bool> function. + +=item null + +A JSON null atom becomes C<undef> in Perl. + +=item shell-style comments (C<< # I<text> >>) + +As a nonstandard extension to the JSON syntax that is enabled by the +C<relaxed> setting, shell-style comments are allowed. They can start +anywhere outside strings and go till the end of the line. + +=back + + +=head2 PERL -> JSON + +The mapping from Perl to JSON is slightly more difficult, as Perl is a +truly typeless language, so we can only guess which JSON type is meant by +a Perl value. + +=over 4 + +=item hash references + +Perl hash references become JSON objects. As there is no inherent +ordering in hash keys (or JSON objects), they will usually be encoded +in a pseudo-random order. This module can optionally sort the hash keys +(determined by the I<canonical> flag), so the same data structure will +serialise to the same JSON text (given same settings and version of +the same backend), but this incurs a runtime overhead and is only rarely useful, +e.g. when you want to compare some JSON text against another for equality. + +=item array references + +Perl array references become JSON arrays. + +=item other references + +Other unblessed references are generally not allowed and will cause an +exception to be thrown, except for references to the integers C<0> and +C<1>, which get turned into C<false> and C<true> atoms in JSON. You can +also use C<JSON::false> and C<JSON::true> to improve readability. + + encode_json [\0,JSON::true] # yields [false,true] + +=item JSON::true, JSON::false, JSON::null + +These special values become JSON true and JSON false values, +respectively. You can also use C<\1> and C<\0> directly if you want. + +=item blessed objects + +Blessed objects are not directly representable in JSON, but C<JSON::XS> +allows various ways of handling objects. See L<OBJECT SERIALISATION>, +below, for details. + +=item simple scalars + +Simple Perl scalars (any scalar that is not a reference) are the most +difficult objects to encode: this module will encode undefined scalars as +JSON C<null> values, scalars that have last been used in a string context +before encoding as JSON strings, and anything else as number value: + + # dump as number + encode_json [2] # yields [2] + encode_json [-3.0e17] # yields [-3e+17] + my $value = 5; encode_json [$value] # yields [5] + + # used as string, so dump as string + print $value; + encode_json [$value] # yields ["5"] + + # undef becomes null + encode_json [undef] # yields [null] + +You can force the type to be a string by stringifying it: + + my $x = 3.1; # some variable containing a number + "$x"; # stringified + $x .= ""; # another, more awkward way to stringify + print $x; # perl does it for you, too, quite often + +You can force the type to be a number by numifying it: + + my $x = "3"; # some variable containing a string + $x += 0; # numify it, ensuring it will be dumped as a number + $x *= 1; # same thing, the choice is yours. + +You can not currently force the type in other, less obscure, ways. Tell me +if you need this capability (but don't forget to explain why it's needed +:). + +Note that numerical precision has the same meaning as under Perl (so +binary to decimal conversion follows the same rules as in Perl, which +can differ to other languages). Also, your perl interpreter might expose +extensions to the floating point numbers of your platform, such as +infinities or NaN's - these cannot be represented in JSON, and it is an +error to pass those in. + +=back + +=head2 OBJECT SERIALISATION + +As for Perl objects, this module only supports a pure JSON representation +(without the ability to deserialise the object automatically again). + +=head3 SERIALISATION + +What happens when this module encounters a Perl object depends on the +C<allow_blessed> and C<convert_blessed> settings, which are used in +this order: + +=over 4 + +=item 1. C<convert_blessed> is enabled and the object has a C<TO_JSON> method. + +In this case, the C<TO_JSON> method of the object is invoked in scalar +context. It must return a single scalar that can be directly encoded into +JSON. This scalar replaces the object in the JSON text. + +For example, the following C<TO_JSON> method will convert all L<URI> +objects to JSON strings when serialised. The fact that these values +originally were L<URI> objects is lost. + + sub URI::TO_JSON { + my ($uri) = @_; + $uri->as_string + } + +=item 2. C<allow_blessed> is enabled. + +The object will be serialised as a JSON null value. + +=item 3. none of the above + +If none of the settings are enabled or the respective methods are missing, +this module throws an exception. + +=back + +=head1 ENCODING/CODESET FLAG NOTES + +This section is taken from JSON::XS. + +The interested reader might have seen a number of flags that signify +encodings or codesets - C<utf8>, C<latin1> and C<ascii>. There seems to be +some confusion on what these do, so here is a short comparison: + +C<utf8> controls whether the JSON text created by C<encode> (and expected +by C<decode>) is UTF-8 encoded or not, while C<latin1> and C<ascii> only +control whether C<encode> escapes character values outside their respective +codeset range. Neither of these flags conflict with each other, although +some combinations make less sense than others. + +Care has been taken to make all flags symmetrical with respect to +C<encode> and C<decode>, that is, texts encoded with any combination of +these flag values will be correctly decoded when the same flags are used +- in general, if you use different flag settings while encoding vs. when +decoding you likely have a bug somewhere. + +Below comes a verbose discussion of these flags. Note that a "codeset" is +simply an abstract set of character-codepoint pairs, while an encoding +takes those codepoint numbers and I<encodes> them, in our case into +octets. Unicode is (among other things) a codeset, UTF-8 is an encoding, +and ISO-8859-1 (= latin 1) and ASCII are both codesets I<and> encodings at +the same time, which can be confusing. + +=over 4 + +=item C<utf8> flag disabled + +When C<utf8> is disabled (the default), then C<encode>/C<decode> generate +and expect Unicode strings, that is, characters with high ordinal Unicode +values (> 255) will be encoded as such characters, and likewise such +characters are decoded as-is, no changes to them will be done, except +"(re-)interpreting" them as Unicode codepoints or Unicode characters, +respectively (to Perl, these are the same thing in strings unless you do +funny/weird/dumb stuff). + +This is useful when you want to do the encoding yourself (e.g. when you +want to have UTF-16 encoded JSON texts) or when some other layer does +the encoding for you (for example, when printing to a terminal using a +filehandle that transparently encodes to UTF-8 you certainly do NOT want +to UTF-8 encode your data first and have Perl encode it another time). + +=item C<utf8> flag enabled + +If the C<utf8>-flag is enabled, C<encode>/C<decode> will encode all +characters using the corresponding UTF-8 multi-byte sequence, and will +expect your input strings to be encoded as UTF-8, that is, no "character" +of the input string must have any value > 255, as UTF-8 does not allow +that. + +The C<utf8> flag therefore switches between two modes: disabled means you +will get a Unicode string in Perl, enabled means you get an UTF-8 encoded +octet/binary string in Perl. + +=item C<latin1> or C<ascii> flags enabled + +With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters +with ordinal values > 255 (> 127 with C<ascii>) and encode the remaining +characters as specified by the C<utf8> flag. + +If C<utf8> is disabled, then the result is also correctly encoded in those +character sets (as both are proper subsets of Unicode, meaning that a +Unicode string with all character values < 256 is the same thing as a +ISO-8859-1 string, and a Unicode string with all character values < 128 is +the same thing as an ASCII string in Perl). + +If C<utf8> is enabled, you still get a correct UTF-8-encoded string, +regardless of these flags, just some more characters will be escaped using +C<\uXXXX> then before. + +Note that ISO-8859-1-I<encoded> strings are not compatible with UTF-8 +encoding, while ASCII-encoded strings are. That is because the ISO-8859-1 +encoding is NOT a subset of UTF-8 (despite the ISO-8859-1 I<codeset> being +a subset of Unicode), while ASCII is. + +Surprisingly, C<decode> will ignore these flags and so treat all input +values as governed by the C<utf8> flag. If it is disabled, this allows you +to decode ISO-8859-1- and ASCII-encoded strings, as both strict subsets of +Unicode. If it is enabled, you can correctly decode UTF-8 encoded strings. + +So neither C<latin1> nor C<ascii> are incompatible with the C<utf8> flag - +they only govern when the JSON output engine escapes a character or not. + +The main use for C<latin1> is to relatively efficiently store binary data +as JSON, at the expense of breaking compatibility with most JSON decoders. + +The main use for C<ascii> is to force the output to not contain characters +with values > 127, which means you can interpret the resulting string +as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and +8-bit-encoding, and still get the same data structure back. This is useful +when your channel for JSON transfer is not 8-bit clean or the encoding +might be mangled in between (e.g. in mail), and works because ASCII is a +proper subset of most 8-bit and multibyte encodings in use in the world. + +=back + +=head1 BACKWARD INCOMPATIBILITY + +Since version 2.90, stringification (and string comparison) for +C<JSON::true> and C<JSON::false> has not been overloaded. It shouldn't +matter as long as you treat them as boolean values, but a code that +expects they are stringified as "true" or "false" doesn't work as +you have expected any more. + + if (JSON::true eq 'true') { # now fails + + print "The result is $JSON::true now."; # => The result is 1 now. + +And now these boolean values don't inherit JSON::Boolean, either. +When you need to test a value is a JSON boolean value or not, use +C<JSON::is_bool> function, instead of testing the value inherits +a particular boolean class or not. + +=head1 BUGS + +Please report bugs on backend selection and additional features +this module provides to RT or GitHub issues for this module: + +=over 4 + +=item https://rt.cpan.org/Public/Dist/Display.html?Queue=JSON + +=item https://github.com/makamaka/JSON/issues + +=back + +Please report bugs and feature requests on decoding/encoding +and boolean behaviors to the author of the backend module you +are using. + +=head1 SEE ALSO + +L<JSON::XS>, L<Cpanel::JSON::XS>, L<JSON::PP> for backends. + +L<JSON::MaybeXS>, an alternative that prefers Cpanel::JSON::XS. + +C<RFC4627>(L<http://www.ietf.org/rfc/rfc4627.txt>) + +=head1 AUTHOR + +Makamaka Hannyaharamitu, E<lt>makamaka[at]cpan.orgE<gt> + +JSON::XS was written by Marc Lehmann <schmorp[at]schmorp.de> + +The release of this new version owes to the courtesy of Marc Lehmann. + + +=head1 COPYRIGHT AND LICENSE + +Copyright 2005-2013 by Makamaka Hannyaharamitu + +This library is free software; you can redistribute it and/or modify +it under the same terms as Perl itself. + +=cut + diff --git a/Master/tlpkg/tlperl/site/lib/JSON/XS.pm b/Master/tlpkg/tlperl/site/lib/JSON/XS.pm new file mode 100644 index 00000000000..674937f178b --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/JSON/XS.pm @@ -0,0 +1,1761 @@ +=head1 NAME + +JSON::XS - JSON serialising/deserialising, done correctly and fast + +=encoding utf-8 + +JSON::XS - 正しくて高速な JSON シリアライザ/デシリアライザ + (http://fleur.hio.jp/perldoc/mix/lib/JSON/XS.html) + +=head1 SYNOPSIS + + use JSON::XS; + + # exported functions, they croak on error + # and expect/generate UTF-8 + + $utf8_encoded_json_text = encode_json $perl_hash_or_arrayref; + $perl_hash_or_arrayref = decode_json $utf8_encoded_json_text; + + # OO-interface + + $coder = JSON::XS->new->ascii->pretty->allow_nonref; + $pretty_printed_unencoded = $coder->encode ($perl_scalar); + $perl_scalar = $coder->decode ($unicode_json_text); + + # Note that JSON version 2.0 and above will automatically use JSON::XS + # if available, at virtually no speed overhead either, so you should + # be able to just: + + use JSON; + + # and do the same things, except that you have a pure-perl fallback now. + +=head1 DESCRIPTION + +This module converts Perl data structures to JSON and vice versa. Its +primary goal is to be I<correct> and its secondary goal is to be +I<fast>. To reach the latter goal it was written in C. + +Beginning with version 2.0 of the JSON module, when both JSON and +JSON::XS are installed, then JSON will fall back on JSON::XS (this can be +overridden) with no overhead due to emulation (by inheriting constructor +and methods). If JSON::XS is not available, it will fall back to the +compatible JSON::PP module as backend, so using JSON instead of JSON::XS +gives you a portable JSON API that can be fast when you need it and +doesn't require a C compiler when that is a problem. + +As this is the n-th-something JSON module on CPAN, what was the reason +to write yet another JSON module? While it seems there are many JSON +modules, none of them correctly handle all corner cases, and in most cases +their maintainers are unresponsive, gone missing, or not listening to bug +reports for other reasons. + +See MAPPING, below, on how JSON::XS maps perl values to JSON values and +vice versa. + +=head2 FEATURES + +=over 4 + +=item * correct Unicode handling + +This module knows how to handle Unicode, documents how and when it does +so, and even documents what "correct" means. + +=item * round-trip integrity + +When you serialise a perl data structure using only data types supported +by JSON and Perl, the deserialised data structure is identical on the Perl +level. (e.g. the string "2.0" doesn't suddenly become "2" just because +it looks like a number). There I<are> minor exceptions to this, read the +MAPPING section below to learn about those. + +=item * strict checking of JSON correctness + +There is no guessing, no generating of illegal JSON texts by default, +and only JSON is accepted as input by default (the latter is a security +feature). + +=item * fast + +Compared to other JSON modules and other serialisers such as Storable, +this module usually compares favourably in terms of speed, too. + +=item * simple to use + +This module has both a simple functional interface as well as an object +oriented interface. + +=item * reasonably versatile output formats + +You can choose between the most compact guaranteed-single-line format +possible (nice for simple line-based protocols), a pure-ASCII format +(for when your transport is not 8-bit clean, still supports the whole +Unicode range), or a pretty-printed format (for when you want to read that +stuff). Or you can combine those features in whatever way you like. + +=back + +=cut + +package JSON::XS; + +use common::sense; + +our $VERSION = 3.04; +our @ISA = qw(Exporter); + +our @EXPORT = qw(encode_json decode_json); + +use Exporter; +use XSLoader; + +use Types::Serialiser (); + +=head1 FUNCTIONAL INTERFACE + +The following convenience methods are provided by this module. They are +exported by default: + +=over 4 + +=item $json_text = encode_json $perl_scalar + +Converts the given Perl data structure to a UTF-8 encoded, binary string +(that is, the string contains octets only). Croaks on error. + +This function call is functionally identical to: + + $json_text = JSON::XS->new->utf8->encode ($perl_scalar) + +Except being faster. + +=item $perl_scalar = decode_json $json_text + +The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries +to parse that as an UTF-8 encoded JSON text, returning the resulting +reference. Croaks on error. + +This function call is functionally identical to: + + $perl_scalar = JSON::XS->new->utf8->decode ($json_text) + +Except being faster. + +=back + + +=head1 A FEW NOTES ON UNICODE AND PERL + +Since this often leads to confusion, here are a few very clear words on +how Unicode works in Perl, modulo bugs. + +=over 4 + +=item 1. Perl strings can store characters with ordinal values > 255. + +This enables you to store Unicode characters as single characters in a +Perl string - very natural. + +=item 2. Perl does I<not> associate an encoding with your strings. + +... until you force it to, e.g. when matching it against a regex, or +printing the scalar to a file, in which case Perl either interprets your +string as locale-encoded text, octets/binary, or as Unicode, depending +on various settings. In no case is an encoding stored together with your +data, it is I<use> that decides encoding, not any magical meta data. + +=item 3. The internal utf-8 flag has no meaning with regards to the +encoding of your string. + +Just ignore that flag unless you debug a Perl bug, a module written in +XS or want to dive into the internals of perl. Otherwise it will only +confuse you, as, despite the name, it says nothing about how your string +is encoded. You can have Unicode strings with that flag set, with that +flag clear, and you can have binary data with that flag set and that flag +clear. Other possibilities exist, too. + +If you didn't know about that flag, just the better, pretend it doesn't +exist. + +=item 4. A "Unicode String" is simply a string where each character can be +validly interpreted as a Unicode code point. + +If you have UTF-8 encoded data, it is no longer a Unicode string, but a +Unicode string encoded in UTF-8, giving you a binary string. + +=item 5. A string containing "high" (> 255) character values is I<not> a UTF-8 string. + +It's a fact. Learn to live with it. + +=back + +I hope this helps :) + + +=head1 OBJECT-ORIENTED INTERFACE + +The object oriented interface lets you configure your own encoding or +decoding style, within the limits of supported formats. + +=over 4 + +=item $json = new JSON::XS + +Creates a new JSON::XS object that can be used to de/encode JSON +strings. All boolean flags described below are by default I<disabled>. + +The mutators for flags all return the JSON object again and thus calls can +be chained: + + my $json = JSON::XS->new->utf8->space_after->encode ({a => [1,2]}) + => {"a": [1, 2]} + +=item $json = $json->ascii ([$enable]) + +=item $enabled = $json->get_ascii + +If C<$enable> is true (or missing), then the C<encode> method will not +generate characters outside the code range C<0..127> (which is ASCII). Any +Unicode characters outside that range will be escaped using either a +single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence, +as per RFC4627. The resulting encoded JSON text can be treated as a native +Unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string, +or any other superset of ASCII. + +If C<$enable> is false, then the C<encode> method will not escape Unicode +characters unless required by the JSON syntax or other flags. This results +in a faster and more compact format. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this +document. + +The main use for this flag is to produce JSON texts that can be +transmitted over a 7-bit channel, as the encoded JSON texts will not +contain any 8 bit characters. + + JSON::XS->new->ascii (1)->encode ([chr 0x10401]) + => ["\ud801\udc01"] + +=item $json = $json->latin1 ([$enable]) + +=item $enabled = $json->get_latin1 + +If C<$enable> is true (or missing), then the C<encode> method will encode +the resulting JSON text as latin1 (or iso-8859-1), escaping any characters +outside the code range C<0..255>. The resulting string can be treated as a +latin1-encoded JSON text or a native Unicode string. The C<decode> method +will not be affected in any way by this flag, as C<decode> by default +expects Unicode, which is a strict superset of latin1. + +If C<$enable> is false, then the C<encode> method will not escape Unicode +characters unless required by the JSON syntax or other flags. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this +document. + +The main use for this flag is efficiently encoding binary data as JSON +text, as most octets will not be escaped, resulting in a smaller encoded +size. The disadvantage is that the resulting JSON text is encoded +in latin1 (and must correctly be treated as such when storing and +transferring), a rare encoding for JSON. It is therefore most useful when +you want to store data structures known to contain binary data efficiently +in files or databases, not when talking to other JSON encoders/decoders. + + JSON::XS->new->latin1->encode (["\x{89}\x{abc}"] + => ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not) + +=item $json = $json->utf8 ([$enable]) + +=item $enabled = $json->get_utf8 + +If C<$enable> is true (or missing), then the C<encode> method will encode +the JSON result into UTF-8, as required by many protocols, while the +C<decode> method expects to be handled an UTF-8-encoded string. Please +note that UTF-8-encoded strings do not contain any characters outside the +range C<0..255>, they are thus useful for bytewise/binary I/O. In future +versions, enabling this option might enable autodetection of the UTF-16 +and UTF-32 encoding families, as described in RFC4627. + +If C<$enable> is false, then the C<encode> method will return the JSON +string as a (non-encoded) Unicode string, while C<decode> expects thus a +Unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs +to be done yourself, e.g. using the Encode module. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this +document. + +Example, output UTF-16BE-encoded JSON: + + use Encode; + $jsontext = encode "UTF-16BE", JSON::XS->new->encode ($object); + +Example, decode UTF-32LE-encoded JSON: + + use Encode; + $object = JSON::XS->new->decode (decode "UTF-32LE", $jsontext); + +=item $json = $json->pretty ([$enable]) + +This enables (or disables) all of the C<indent>, C<space_before> and +C<space_after> (and in the future possibly more) flags in one call to +generate the most readable (or most compact) form possible. + +Example, pretty-print some simple structure: + + my $json = JSON::XS->new->pretty(1)->encode ({a => [1,2]}) + => + { + "a" : [ + 1, + 2 + ] + } + +=item $json = $json->indent ([$enable]) + +=item $enabled = $json->get_indent + +If C<$enable> is true (or missing), then the C<encode> method will use a multiline +format as output, putting every array member or object/hash key-value pair +into its own line, indenting them properly. + +If C<$enable> is false, no newlines or indenting will be produced, and the +resulting JSON text is guaranteed not to contain any C<newlines>. + +This setting has no effect when decoding JSON texts. + +=item $json = $json->space_before ([$enable]) + +=item $enabled = $json->get_space_before + +If C<$enable> is true (or missing), then the C<encode> method will add an extra +optional space before the C<:> separating keys from values in JSON objects. + +If C<$enable> is false, then the C<encode> method will not add any extra +space at those places. + +This setting has no effect when decoding JSON texts. You will also +most likely combine this setting with C<space_after>. + +Example, space_before enabled, space_after and indent disabled: + + {"key" :"value"} + +=item $json = $json->space_after ([$enable]) + +=item $enabled = $json->get_space_after + +If C<$enable> is true (or missing), then the C<encode> method will add an extra +optional space after the C<:> separating keys from values in JSON objects +and extra whitespace after the C<,> separating key-value pairs and array +members. + +If C<$enable> is false, then the C<encode> method will not add any extra +space at those places. + +This setting has no effect when decoding JSON texts. + +Example, space_before and indent disabled, space_after enabled: + + {"key": "value"} + +=item $json = $json->relaxed ([$enable]) + +=item $enabled = $json->get_relaxed + +If C<$enable> is true (or missing), then C<decode> will accept some +extensions to normal JSON syntax (see below). C<encode> will not be +affected in anyway. I<Be aware that this option makes you accept invalid +JSON texts as if they were valid!>. I suggest only to use this option to +parse application-specific files written by humans (configuration files, +resource files etc.) + +If C<$enable> is false (the default), then C<decode> will only accept +valid JSON texts. + +Currently accepted extensions are: + +=over 4 + +=item * list items can have an end-comma + +JSON I<separates> array elements and key-value pairs with commas. This +can be annoying if you write JSON texts manually and want to be able to +quickly append elements, so this extension accepts comma at the end of +such items not just between them: + + [ + 1, + 2, <- this comma not normally allowed + ] + { + "k1": "v1", + "k2": "v2", <- this comma not normally allowed + } + +=item * shell-style '#'-comments + +Whenever JSON allows whitespace, shell-style comments are additionally +allowed. They are terminated by the first carriage-return or line-feed +character, after which more white-space and comments are allowed. + + [ + 1, # this comment not allowed in JSON + # neither this one... + ] + +=item * literal ASCII TAB characters in strings + +Literal ASCII TAB characters are now allowed in strings (and treated as +C<\t>). + + [ + "Hello\tWorld", + "Hello<TAB>World", # literal <TAB> would not normally be allowed + ] + +=back + +=item $json = $json->canonical ([$enable]) + +=item $enabled = $json->get_canonical + +If C<$enable> is true (or missing), then the C<encode> method will output JSON objects +by sorting their keys. This is adding a comparatively high overhead. + +If C<$enable> is false, then the C<encode> method will output key-value +pairs in the order Perl stores them (which will likely change between runs +of the same script, and can change even within the same run from 5.18 +onwards). + +This option is useful if you want the same data structure to be encoded as +the same JSON text (given the same overall settings). If it is disabled, +the same hash might be encoded differently even if contains the same data, +as key-value pairs have no inherent ordering in Perl. + +This setting has no effect when decoding JSON texts. + +This setting has currently no effect on tied hashes. + +=item $json = $json->allow_nonref ([$enable]) + +=item $enabled = $json->get_allow_nonref + +If C<$enable> is true (or missing), then the C<encode> method can convert a +non-reference into its corresponding string, number or null JSON value, +which is an extension to RFC4627. Likewise, C<decode> will accept those JSON +values instead of croaking. + +If C<$enable> is false, then the C<encode> method will croak if it isn't +passed an arrayref or hashref, as JSON texts must either be an object +or array. Likewise, C<decode> will croak if given something that is not a +JSON object or array. + +Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, +resulting in an invalid JSON text: + + JSON::XS->new->allow_nonref->encode ("Hello, World!") + => "Hello, World!" + +=item $json = $json->allow_unknown ([$enable]) + +=item $enabled = $json->get_allow_unknown + +If C<$enable> is true (or missing), then C<encode> will I<not> throw an +exception when it encounters values it cannot represent in JSON (for +example, filehandles) but instead will encode a JSON C<null> value. Note +that blessed objects are not included here and are handled separately by +c<allow_nonref>. + +If C<$enable> is false (the default), then C<encode> will throw an +exception when it encounters anything it cannot encode as JSON. + +This option does not affect C<decode> in any way, and it is recommended to +leave it off unless you know your communications partner. + +=item $json = $json->allow_blessed ([$enable]) + +=item $enabled = $json->get_allow_blessed + +See L<OBJECT SERIALISATION> for details. + +If C<$enable> is true (or missing), then the C<encode> method will not +barf when it encounters a blessed reference that it cannot convert +otherwise. Instead, a JSON C<null> value is encoded instead of the object. + +If C<$enable> is false (the default), then C<encode> will throw an +exception when it encounters a blessed object that it cannot convert +otherwise. + +This setting has no effect on C<decode>. + +=item $json = $json->convert_blessed ([$enable]) + +=item $enabled = $json->get_convert_blessed + +See L<OBJECT SERIALISATION> for details. + +If C<$enable> is true (or missing), then C<encode>, upon encountering a +blessed object, will check for the availability of the C<TO_JSON> method +on the object's class. If found, it will be called in scalar context and +the resulting scalar will be encoded instead of the object. + +The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON> +returns other blessed objects, those will be handled in the same +way. C<TO_JSON> must take care of not causing an endless recursion cycle +(== crash) in this case. The name of C<TO_JSON> was chosen because other +methods called by the Perl core (== not by the user of the object) are +usually in upper case letters and to avoid collisions with any C<to_json> +function or method. + +If C<$enable> is false (the default), then C<encode> will not consider +this type of conversion. + +This setting has no effect on C<decode>. + +=item $json = $json->allow_tags ([$enable]) + +=item $enabled = $json->allow_tags + +See L<OBJECT SERIALISATION> for details. + +If C<$enable> is true (or missing), then C<encode>, upon encountering a +blessed object, will check for the availability of the C<FREEZE> method on +the object's class. If found, it will be used to serialise the object into +a nonstandard tagged JSON value (that JSON decoders cannot decode). + +It also causes C<decode> to parse such tagged JSON values and deserialise +them via a call to the C<THAW> method. + +If C<$enable> is false (the default), then C<encode> will not consider +this type of conversion, and tagged JSON values will cause a parse error +in C<decode>, as if tags were not part of the grammar. + +=item $json = $json->filter_json_object ([$coderef->($hashref)]) + +When C<$coderef> is specified, it will be called from C<decode> each +time it decodes a JSON object. The only argument is a reference to the +newly-created hash. If the code references returns a single scalar (which +need not be a reference), this value (i.e. a copy of that scalar to avoid +aliasing) is inserted into the deserialised data structure. If it returns +an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the +original deserialised hash will be inserted. This setting can slow down +decoding considerably. + +When C<$coderef> is omitted or undefined, any existing callback will +be removed and C<decode> will not change the deserialised hash in any +way. + +Example, convert all JSON objects into the integer 5: + + my $js = JSON::XS->new->filter_json_object (sub { 5 }); + # returns [5] + $js->decode ('[{}]') + # throw an exception because allow_nonref is not enabled + # so a lone 5 is not allowed. + $js->decode ('{"a":1, "b":2}'); + +=item $json = $json->filter_json_single_key_object ($key [=> $coderef->($value)]) + +Works remotely similar to C<filter_json_object>, but is only called for +JSON objects having a single key named C<$key>. + +This C<$coderef> is called before the one specified via +C<filter_json_object>, if any. It gets passed the single value in the JSON +object. If it returns a single value, it will be inserted into the data +structure. If it returns nothing (not even C<undef> but the empty list), +the callback from C<filter_json_object> will be called next, as if no +single-key callback were specified. + +If C<$coderef> is omitted or undefined, the corresponding callback will be +disabled. There can only ever be one callback for a given key. + +As this callback gets called less often then the C<filter_json_object> +one, decoding speed will not usually suffer as much. Therefore, single-key +objects make excellent targets to serialise Perl objects into, especially +as single-key JSON objects are as close to the type-tagged value concept +as JSON gets (it's basically an ID/VALUE tuple). Of course, JSON does not +support this in any way, so you need to make sure your data never looks +like a serialised Perl hash. + +Typical names for the single object key are C<__class_whatever__>, or +C<$__dollars_are_rarely_used__$> or C<}ugly_brace_placement>, or even +things like C<__class_md5sum(classname)__>, to reduce the risk of clashing +with real hashes. + +Example, decode JSON objects of the form C<< { "__widget__" => <id> } >> +into the corresponding C<< $WIDGET{<id>} >> object: + + # return whatever is in $WIDGET{5}: + JSON::XS + ->new + ->filter_json_single_key_object (__widget__ => sub { + $WIDGET{ $_[0] } + }) + ->decode ('{"__widget__": 5') + + # this can be used with a TO_JSON method in some "widget" class + # for serialisation to json: + sub WidgetBase::TO_JSON { + my ($self) = @_; + + unless ($self->{id}) { + $self->{id} = ..get..some..id..; + $WIDGET{$self->{id}} = $self; + } + + { __widget__ => $self->{id} } + } + +=item $json = $json->shrink ([$enable]) + +=item $enabled = $json->get_shrink + +Perl usually over-allocates memory a bit when allocating space for +strings. This flag optionally resizes strings generated by either +C<encode> or C<decode> to their minimum size possible. This can save +memory when your JSON texts are either very very long or you have many +short strings. It will also try to downgrade any strings to octet-form +if possible: perl stores strings internally either in an encoding called +UTF-X or in octet-form. The latter cannot store everything but uses less +space in general (and some buggy Perl or C code might even rely on that +internal representation being used). + +The actual definition of what shrink does might change in future versions, +but it will always try to save space at the expense of time. + +If C<$enable> is true (or missing), the string returned by C<encode> will +be shrunk-to-fit, while all strings generated by C<decode> will also be +shrunk-to-fit. + +If C<$enable> is false, then the normal perl allocation algorithms are used. +If you work with your data, then this is likely to be faster. + +In the future, this setting might control other things, such as converting +strings that look like integers or floats into integers or floats +internally (there is no difference on the Perl level), saving space. + +=item $json = $json->max_depth ([$maximum_nesting_depth]) + +=item $max_depth = $json->get_max_depth + +Sets the maximum nesting level (default C<512>) accepted while encoding +or decoding. If a higher nesting level is detected in JSON text or a Perl +data structure, then the encoder and decoder will stop and croak at that +point. + +Nesting level is defined by number of hash- or arrayrefs that the encoder +needs to traverse to reach a given point or the number of C<{> or C<[> +characters without their matching closing parenthesis crossed to reach a +given character in a string. + +Setting the maximum depth to one disallows any nesting, so that ensures +that the object is only a single hash/object or array. + +If no argument is given, the highest possible setting will be used, which +is rarely useful. + +Note that nesting is implemented by recursion in C. The default value has +been chosen to be as large as typical operating systems allow without +crashing. + +See SECURITY CONSIDERATIONS, below, for more info on why this is useful. + +=item $json = $json->max_size ([$maximum_string_size]) + +=item $max_size = $json->get_max_size + +Set the maximum length a JSON text may have (in bytes) where decoding is +being attempted. The default is C<0>, meaning no limit. When C<decode> +is called on a string that is longer then this many bytes, it will not +attempt to decode the string but throw an exception. This setting has no +effect on C<encode> (yet). + +If no argument is given, the limit check will be deactivated (same as when +C<0> is specified). + +See SECURITY CONSIDERATIONS, below, for more info on why this is useful. + +=item $json_text = $json->encode ($perl_scalar) + +Converts the given Perl value or data structure to its JSON +representation. Croaks on error. + +=item $perl_scalar = $json->decode ($json_text) + +The opposite of C<encode>: expects a JSON text and tries to parse it, +returning the resulting simple scalar or reference. Croaks on error. + +=item ($perl_scalar, $characters) = $json->decode_prefix ($json_text) + +This works like the C<decode> method, but instead of raising an exception +when there is trailing garbage after the first JSON object, it will +silently stop parsing there and return the number of characters consumed +so far. + +This is useful if your JSON texts are not delimited by an outer protocol +and you need to know where the JSON text ends. + + JSON::XS->new->decode_prefix ("[1] the tail") + => ([1], 3) + +=back + + +=head1 INCREMENTAL PARSING + +In some cases, there is the need for incremental parsing of JSON +texts. While this module always has to keep both JSON text and resulting +Perl data structure in memory at one time, it does allow you to parse a +JSON stream incrementally. It does so by accumulating text until it has +a full JSON object, which it then can decode. This process is similar to +using C<decode_prefix> to see if a full JSON object is available, but +is much more efficient (and can be implemented with a minimum of method +calls). + +JSON::XS will only attempt to parse the JSON text once it is sure it +has enough text to get a decisive result, using a very simple but +truly incremental parser. This means that it sometimes won't stop as +early as the full parser, for example, it doesn't detect mismatched +parentheses. The only thing it guarantees is that it starts decoding as +soon as a syntactically valid JSON text has been seen. This means you need +to set resource limits (e.g. C<max_size>) to ensure the parser will stop +parsing in the presence if syntax errors. + +The following methods implement this incremental parser. + +=over 4 + +=item [void, scalar or list context] = $json->incr_parse ([$string]) + +This is the central parsing function. It can both append new text and +extract objects from the stream accumulated so far (both of these +functions are optional). + +If C<$string> is given, then this string is appended to the already +existing JSON fragment stored in the C<$json> object. + +After that, if the function is called in void context, it will simply +return without doing anything further. This can be used to add more text +in as many chunks as you want. + +If the method is called in scalar context, then it will try to extract +exactly I<one> JSON object. If that is successful, it will return this +object, otherwise it will return C<undef>. If there is a parse error, +this method will croak just as C<decode> would do (one can then use +C<incr_skip> to skip the erroneous part). This is the most common way of +using the method. + +And finally, in list context, it will try to extract as many objects +from the stream as it can find and return them, or the empty list +otherwise. For this to work, there must be no separators (other than +whitespace) between the JSON objects or arrays, instead they must be +concatenated back-to-back. If an error occurs, an exception will be +raised as in the scalar context case. Note that in this case, any +previously-parsed JSON texts will be lost. + +Example: Parse some JSON arrays/objects in a given string and return +them. + + my @objs = JSON::XS->new->incr_parse ("[5][7][1,2]"); + +=item $lvalue_string = $json->incr_text + +This method returns the currently stored JSON fragment as an lvalue, that +is, you can manipulate it. This I<only> works when a preceding call to +C<incr_parse> in I<scalar context> successfully returned an object. Under +all other circumstances you must not call this function (I mean it. +although in simple tests it might actually work, it I<will> fail under +real world conditions). As a special exception, you can also call this +method before having parsed anything. + +That means you can only use this function to look at or manipulate text +before or after complete JSON objects, not while the parser is in the +middle of parsing a JSON object. + +This function is useful in two cases: a) finding the trailing text after a +JSON object or b) parsing multiple JSON objects separated by non-JSON text +(such as commas). + +=item $json->incr_skip + +This will reset the state of the incremental parser and will remove +the parsed text from the input buffer so far. This is useful after +C<incr_parse> died, in which case the input buffer and incremental parser +state is left unchanged, to skip the text parsed so far and to reset the +parse state. + +The difference to C<incr_reset> is that only text until the parse error +occurred is removed. + +=item $json->incr_reset + +This completely resets the incremental parser, that is, after this call, +it will be as if the parser had never parsed anything. + +This is useful if you want to repeatedly parse JSON objects and want to +ignore any trailing data, which means you have to reset the parser after +each successful decode. + +=back + +=head2 LIMITATIONS + +All options that affect decoding are supported, except +C<allow_nonref>. The reason for this is that it cannot be made to work +sensibly: JSON objects and arrays are self-delimited, i.e. you can +concatenate them back to back and still decode them perfectly. This does +not hold true for JSON numbers, however. + +For example, is the string C<1> a single JSON number, or is it simply the +start of C<12>? Or is C<12> a single JSON number, or the concatenation +of C<1> and C<2>? In neither case you can tell, and this is why JSON::XS +takes the conservative route and disallows this case. + +=head2 EXAMPLES + +Some examples will make all this clearer. First, a simple example that +works similarly to C<decode_prefix>: We want to decode the JSON object at +the start of a string and identify the portion after the JSON object: + + my $text = "[1,2,3] hello"; + + my $json = new JSON::XS; + + my $obj = $json->incr_parse ($text) + or die "expected JSON object or array at beginning of string"; + + my $tail = $json->incr_text; + # $tail now contains " hello" + +Easy, isn't it? + +Now for a more complicated example: Imagine a hypothetical protocol where +you read some requests from a TCP stream, and each request is a JSON +array, without any separation between them (in fact, it is often useful to +use newlines as "separators", as these get interpreted as whitespace at +the start of the JSON text, which makes it possible to test said protocol +with C<telnet>...). + +Here is how you'd do it (it is trivial to write this in an event-based +manner): + + my $json = new JSON::XS; + + # read some data from the socket + while (sysread $socket, my $buf, 4096) { + + # split and decode as many requests as possible + for my $request ($json->incr_parse ($buf)) { + # act on the $request + } + } + +Another complicated example: Assume you have a string with JSON objects +or arrays, all separated by (optional) comma characters (e.g. C<[1],[2], +[3]>). To parse them, we have to skip the commas between the JSON texts, +and here is where the lvalue-ness of C<incr_text> comes in useful: + + my $text = "[1],[2], [3]"; + my $json = new JSON::XS; + + # void context, so no parsing done + $json->incr_parse ($text); + + # now extract as many objects as possible. note the + # use of scalar context so incr_text can be called. + while (my $obj = $json->incr_parse) { + # do something with $obj + + # now skip the optional comma + $json->incr_text =~ s/^ \s* , //x; + } + +Now lets go for a very complex example: Assume that you have a gigantic +JSON array-of-objects, many gigabytes in size, and you want to parse it, +but you cannot load it into memory fully (this has actually happened in +the real world :). + +Well, you lost, you have to implement your own JSON parser. But JSON::XS +can still help you: You implement a (very simple) array parser and let +JSON decode the array elements, which are all full JSON objects on their +own (this wouldn't work if the array elements could be JSON numbers, for +example): + + my $json = new JSON::XS; + + # open the monster + open my $fh, "<bigfile.json" + or die "bigfile: $!"; + + # first parse the initial "[" + for (;;) { + sysread $fh, my $buf, 65536 + or die "read error: $!"; + $json->incr_parse ($buf); # void context, so no parsing + + # Exit the loop once we found and removed(!) the initial "[". + # In essence, we are (ab-)using the $json object as a simple scalar + # we append data to. + last if $json->incr_text =~ s/^ \s* \[ //x; + } + + # now we have the skipped the initial "[", so continue + # parsing all the elements. + for (;;) { + # in this loop we read data until we got a single JSON object + for (;;) { + if (my $obj = $json->incr_parse) { + # do something with $obj + last; + } + + # add more data + sysread $fh, my $buf, 65536 + or die "read error: $!"; + $json->incr_parse ($buf); # void context, so no parsing + } + + # in this loop we read data until we either found and parsed the + # separating "," between elements, or the final "]" + for (;;) { + # first skip whitespace + $json->incr_text =~ s/^\s*//; + + # if we find "]", we are done + if ($json->incr_text =~ s/^\]//) { + print "finished.\n"; + exit; + } + + # if we find ",", we can continue with the next element + if ($json->incr_text =~ s/^,//) { + last; + } + + # if we find anything else, we have a parse error! + if (length $json->incr_text) { + die "parse error near ", $json->incr_text; + } + + # else add more data + sysread $fh, my $buf, 65536 + or die "read error: $!"; + $json->incr_parse ($buf); # void context, so no parsing + } + +This is a complex example, but most of the complexity comes from the fact +that we are trying to be correct (bear with me if I am wrong, I never ran +the above example :). + + + +=head1 MAPPING + +This section describes how JSON::XS maps Perl values to JSON values and +vice versa. These mappings are designed to "do the right thing" in most +circumstances automatically, preserving round-tripping characteristics +(what you put in comes out as something equivalent). + +For the more enlightened: note that in the following descriptions, +lowercase I<perl> refers to the Perl interpreter, while uppercase I<Perl> +refers to the abstract Perl language itself. + + +=head2 JSON -> PERL + +=over 4 + +=item object + +A JSON object becomes a reference to a hash in Perl. No ordering of object +keys is preserved (JSON does not preserve object key ordering itself). + +=item array + +A JSON array becomes a reference to an array in Perl. + +=item string + +A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON +are represented by the same codepoints in the Perl string, so no manual +decoding is necessary. + +=item number + +A JSON number becomes either an integer, numeric (floating point) or +string scalar in perl, depending on its range and any fractional parts. On +the Perl level, there is no difference between those as Perl handles all +the conversion details, but an integer may take slightly less memory and +might represent more values exactly than floating point numbers. + +If the number consists of digits only, JSON::XS will try to represent +it as an integer value. If that fails, it will try to represent it as +a numeric (floating point) value if that is possible without loss of +precision. Otherwise it will preserve the number as a string value (in +which case you lose roundtripping ability, as the JSON number will be +re-encoded to a JSON string). + +Numbers containing a fractional or exponential part will always be +represented as numeric (floating point) values, possibly at a loss of +precision (in which case you might lose perfect roundtripping ability, but +the JSON number will still be re-encoded as a JSON number). + +Note that precision is not accuracy - binary floating point values cannot +represent most decimal fractions exactly, and when converting from and to +floating point, JSON::XS only guarantees precision up to but not including +the least significant bit. + +=item true, false + +These JSON atoms become C<Types::Serialiser::true> and +C<Types::Serialiser::false>, respectively. They are overloaded to act +almost exactly like the numbers C<1> and C<0>. You can check whether +a scalar is a JSON boolean by using the C<Types::Serialiser::is_bool> +function (after C<use Types::Serialier>, of course). + +=item null + +A JSON null atom becomes C<undef> in Perl. + +=item shell-style comments (C<< # I<text> >>) + +As a nonstandard extension to the JSON syntax that is enabled by the +C<relaxed> setting, shell-style comments are allowed. They can start +anywhere outside strings and go till the end of the line. + +=item tagged values (C<< (I<tag>)I<value> >>). + +Another nonstandard extension to the JSON syntax, enabled with the +C<allow_tags> setting, are tagged values. In this implementation, the +I<tag> must be a perl package/class name encoded as a JSON string, and the +I<value> must be a JSON array encoding optional constructor arguments. + +See L<OBJECT SERIALISATION>, below, for details. + +=back + + +=head2 PERL -> JSON + +The mapping from Perl to JSON is slightly more difficult, as Perl is a +truly typeless language, so we can only guess which JSON type is meant by +a Perl value. + +=over 4 + +=item hash references + +Perl hash references become JSON objects. As there is no inherent +ordering in hash keys (or JSON objects), they will usually be encoded +in a pseudo-random order. JSON::XS can optionally sort the hash keys +(determined by the I<canonical> flag), so the same datastructure will +serialise to the same JSON text (given same settings and version of +JSON::XS), but this incurs a runtime overhead and is only rarely useful, +e.g. when you want to compare some JSON text against another for equality. + +=item array references + +Perl array references become JSON arrays. + +=item other references + +Other unblessed references are generally not allowed and will cause an +exception to be thrown, except for references to the integers C<0> and +C<1>, which get turned into C<false> and C<true> atoms in JSON. + +Since C<JSON::XS> uses the boolean model from L<Types::Serialiser>, you +can also C<use Types::Serialiser> and then use C<Types::Serialiser::false> +and C<Types::Serialiser::true> to improve readability. + + use Types::Serialiser; + encode_json [\0, Types::Serialiser::true] # yields [false,true] + +=item Types::Serialiser::true, Types::Serialiser::false + +These special values from the L<Types::Serialiser> module become JSON true +and JSON false values, respectively. You can also use C<\1> and C<\0> +directly if you want. + +=item blessed objects + +Blessed objects are not directly representable in JSON, but C<JSON::XS> +allows various ways of handling objects. See L<OBJECT SERIALISATION>, +below, for details. + +=item simple scalars + +Simple Perl scalars (any scalar that is not a reference) are the most +difficult objects to encode: JSON::XS will encode undefined scalars as +JSON C<null> values, scalars that have last been used in a string context +before encoding as JSON strings, and anything else as number value: + + # dump as number + encode_json [2] # yields [2] + encode_json [-3.0e17] # yields [-3e+17] + my $value = 5; encode_json [$value] # yields [5] + + # used as string, so dump as string + print $value; + encode_json [$value] # yields ["5"] + + # undef becomes null + encode_json [undef] # yields [null] + +You can force the type to be a JSON string by stringifying it: + + my $x = 3.1; # some variable containing a number + "$x"; # stringified + $x .= ""; # another, more awkward way to stringify + print $x; # perl does it for you, too, quite often + +You can force the type to be a JSON number by numifying it: + + my $x = "3"; # some variable containing a string + $x += 0; # numify it, ensuring it will be dumped as a number + $x *= 1; # same thing, the choice is yours. + +You can not currently force the type in other, less obscure, ways. Tell me +if you need this capability (but don't forget to explain why it's needed +:). + +Note that numerical precision has the same meaning as under Perl (so +binary to decimal conversion follows the same rules as in Perl, which +can differ to other languages). Also, your perl interpreter might expose +extensions to the floating point numbers of your platform, such as +infinities or NaN's - these cannot be represented in JSON, and it is an +error to pass those in. + +=back + +=head2 OBJECT SERIALISATION + +As JSON cannot directly represent Perl objects, you have to choose between +a pure JSON representation (without the ability to deserialise the object +automatically again), and a nonstandard extension to the JSON syntax, +tagged values. + +=head3 SERIALISATION + +What happens when C<JSON::XS> encounters a Perl object depends on the +C<allow_blessed>, C<convert_blessed> and C<allow_tags> settings, which are +used in this order: + +=over 4 + +=item 1. C<allow_tags> is enabled and the object has a C<FREEZE> method. + +In this case, C<JSON::XS> uses the L<Types::Serialiser> object +serialisation protocol to create a tagged JSON value, using a nonstandard +extension to the JSON syntax. + +This works by invoking the C<FREEZE> method on the object, with the first +argument being the object to serialise, and the second argument being the +constant string C<JSON> to distinguish it from other serialisers. + +The C<FREEZE> method can return any number of values (i.e. zero or +more). These values and the paclkage/classname of the object will then be +encoded as a tagged JSON value in the following format: + + ("classname")[FREEZE return values...] + +e.g.: + + ("URI")["http://www.google.com/"] + ("MyDate")[2013,10,29] + ("ImageData::JPEG")["Z3...VlCg=="] + +For example, the hypothetical C<My::Object> C<FREEZE> method might use the +objects C<type> and C<id> members to encode the object: + + sub My::Object::FREEZE { + my ($self, $serialiser) = @_; + + ($self->{type}, $self->{id}) + } + +=item 2. C<convert_blessed> is enabled and the object has a C<TO_JSON> method. + +In this case, the C<TO_JSON> method of the object is invoked in scalar +context. It must return a single scalar that can be directly encoded into +JSON. This scalar replaces the object in the JSON text. + +For example, the following C<TO_JSON> method will convert all L<URI> +objects to JSON strings when serialised. The fatc that these values +originally were L<URI> objects is lost. + + sub URI::TO_JSON { + my ($uri) = @_; + $uri->as_string + } + +=item 3. C<allow_blessed> is enabled. + +The object will be serialised as a JSON null value. + +=item 4. none of the above + +If none of the settings are enabled or the respective methods are missing, +C<JSON::XS> throws an exception. + +=back + +=head3 DESERIALISATION + +For deserialisation there are only two cases to consider: either +nonstandard tagging was used, in which case C<allow_tags> decides, +or objects cannot be automatically be deserialised, in which +case you can use postprocessing or the C<filter_json_object> or +C<filter_json_single_key_object> callbacks to get some real objects our of +your JSON. + +This section only considers the tagged value case: I a tagged JSON object +is encountered during decoding and C<allow_tags> is disabled, a parse +error will result (as if tagged values were not part of the grammar). + +If C<allow_tags> is enabled, C<JSON::XS> will look up the C<THAW> method +of the package/classname used during serialisation (it will not attempt +to load the package as a Perl module). If there is no such method, the +decoding will fail with an error. + +Otherwise, the C<THAW> method is invoked with the classname as first +argument, the constant string C<JSON> as second argument, and all the +values from the JSON array (the values originally returned by the +C<FREEZE> method) as remaining arguments. + +The method must then return the object. While technically you can return +any Perl scalar, you might have to enable the C<enable_nonref> setting to +make that work in all cases, so better return an actual blessed reference. + +As an example, let's implement a C<THAW> function that regenerates the +C<My::Object> from the C<FREEZE> example earlier: + + sub My::Object::THAW { + my ($class, $serialiser, $type, $id) = @_; + + $class->new (type => $type, id => $id) + } + + +=head1 ENCODING/CODESET FLAG NOTES + +The interested reader might have seen a number of flags that signify +encodings or codesets - C<utf8>, C<latin1> and C<ascii>. There seems to be +some confusion on what these do, so here is a short comparison: + +C<utf8> controls whether the JSON text created by C<encode> (and expected +by C<decode>) is UTF-8 encoded or not, while C<latin1> and C<ascii> only +control whether C<encode> escapes character values outside their respective +codeset range. Neither of these flags conflict with each other, although +some combinations make less sense than others. + +Care has been taken to make all flags symmetrical with respect to +C<encode> and C<decode>, that is, texts encoded with any combination of +these flag values will be correctly decoded when the same flags are used +- in general, if you use different flag settings while encoding vs. when +decoding you likely have a bug somewhere. + +Below comes a verbose discussion of these flags. Note that a "codeset" is +simply an abstract set of character-codepoint pairs, while an encoding +takes those codepoint numbers and I<encodes> them, in our case into +octets. Unicode is (among other things) a codeset, UTF-8 is an encoding, +and ISO-8859-1 (= latin 1) and ASCII are both codesets I<and> encodings at +the same time, which can be confusing. + +=over 4 + +=item C<utf8> flag disabled + +When C<utf8> is disabled (the default), then C<encode>/C<decode> generate +and expect Unicode strings, that is, characters with high ordinal Unicode +values (> 255) will be encoded as such characters, and likewise such +characters are decoded as-is, no changes to them will be done, except +"(re-)interpreting" them as Unicode codepoints or Unicode characters, +respectively (to Perl, these are the same thing in strings unless you do +funny/weird/dumb stuff). + +This is useful when you want to do the encoding yourself (e.g. when you +want to have UTF-16 encoded JSON texts) or when some other layer does +the encoding for you (for example, when printing to a terminal using a +filehandle that transparently encodes to UTF-8 you certainly do NOT want +to UTF-8 encode your data first and have Perl encode it another time). + +=item C<utf8> flag enabled + +If the C<utf8>-flag is enabled, C<encode>/C<decode> will encode all +characters using the corresponding UTF-8 multi-byte sequence, and will +expect your input strings to be encoded as UTF-8, that is, no "character" +of the input string must have any value > 255, as UTF-8 does not allow +that. + +The C<utf8> flag therefore switches between two modes: disabled means you +will get a Unicode string in Perl, enabled means you get an UTF-8 encoded +octet/binary string in Perl. + +=item C<latin1> or C<ascii> flags enabled + +With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters +with ordinal values > 255 (> 127 with C<ascii>) and encode the remaining +characters as specified by the C<utf8> flag. + +If C<utf8> is disabled, then the result is also correctly encoded in those +character sets (as both are proper subsets of Unicode, meaning that a +Unicode string with all character values < 256 is the same thing as a +ISO-8859-1 string, and a Unicode string with all character values < 128 is +the same thing as an ASCII string in Perl). + +If C<utf8> is enabled, you still get a correct UTF-8-encoded string, +regardless of these flags, just some more characters will be escaped using +C<\uXXXX> then before. + +Note that ISO-8859-1-I<encoded> strings are not compatible with UTF-8 +encoding, while ASCII-encoded strings are. That is because the ISO-8859-1 +encoding is NOT a subset of UTF-8 (despite the ISO-8859-1 I<codeset> being +a subset of Unicode), while ASCII is. + +Surprisingly, C<decode> will ignore these flags and so treat all input +values as governed by the C<utf8> flag. If it is disabled, this allows you +to decode ISO-8859-1- and ASCII-encoded strings, as both strict subsets of +Unicode. If it is enabled, you can correctly decode UTF-8 encoded strings. + +So neither C<latin1> nor C<ascii> are incompatible with the C<utf8> flag - +they only govern when the JSON output engine escapes a character or not. + +The main use for C<latin1> is to relatively efficiently store binary data +as JSON, at the expense of breaking compatibility with most JSON decoders. + +The main use for C<ascii> is to force the output to not contain characters +with values > 127, which means you can interpret the resulting string +as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and +8-bit-encoding, and still get the same data structure back. This is useful +when your channel for JSON transfer is not 8-bit clean or the encoding +might be mangled in between (e.g. in mail), and works because ASCII is a +proper subset of most 8-bit and multibyte encodings in use in the world. + +=back + + +=head2 JSON and ECMAscript + +JSON syntax is based on how literals are represented in javascript (the +not-standardised predecessor of ECMAscript) which is presumably why it is +called "JavaScript Object Notation". + +However, JSON is not a subset (and also not a superset of course) of +ECMAscript (the standard) or javascript (whatever browsers actually +implement). + +If you want to use javascript's C<eval> function to "parse" JSON, you +might run into parse errors for valid JSON texts, or the resulting data +structure might not be queryable: + +One of the problems is that U+2028 and U+2029 are valid characters inside +JSON strings, but are not allowed in ECMAscript string literals, so the +following Perl fragment will not output something that can be guaranteed +to be parsable by javascript's C<eval>: + + use JSON::XS; + + print encode_json [chr 0x2028]; + +The right fix for this is to use a proper JSON parser in your javascript +programs, and not rely on C<eval> (see for example Douglas Crockford's +F<json2.js> parser). + +If this is not an option, you can, as a stop-gap measure, simply encode to +ASCII-only JSON: + + use JSON::XS; + + print JSON::XS->new->ascii->encode ([chr 0x2028]); + +Note that this will enlarge the resulting JSON text quite a bit if you +have many non-ASCII characters. You might be tempted to run some regexes +to only escape U+2028 and U+2029, e.g.: + + # DO NOT USE THIS! + my $json = JSON::XS->new->utf8->encode ([chr 0x2028]); + $json =~ s/\xe2\x80\xa8/\\u2028/g; # escape U+2028 + $json =~ s/\xe2\x80\xa9/\\u2029/g; # escape U+2029 + print $json; + +Note that I<this is a bad idea>: the above only works for U+2028 and +U+2029 and thus only for fully ECMAscript-compliant parsers. Many existing +javascript implementations, however, have issues with other characters as +well - using C<eval> naively simply I<will> cause problems. + +Another problem is that some javascript implementations reserve +some property names for their own purposes (which probably makes +them non-ECMAscript-compliant). For example, Iceweasel reserves the +C<__proto__> property name for its own purposes. + +If that is a problem, you could parse try to filter the resulting JSON +output for these property strings, e.g.: + + $json =~ s/"__proto__"\s*:/"__proto__renamed":/g; + +This works because C<__proto__> is not valid outside of strings, so every +occurrence of C<"__proto__"\s*:> must be a string used as property name. + +If you know of other incompatibilities, please let me know. + + +=head2 JSON and YAML + +You often hear that JSON is a subset of YAML. This is, however, a mass +hysteria(*) and very far from the truth (as of the time of this writing), +so let me state it clearly: I<in general, there is no way to configure +JSON::XS to output a data structure as valid YAML> that works in all +cases. + +If you really must use JSON::XS to generate YAML, you should use this +algorithm (subject to change in future versions): + + my $to_yaml = JSON::XS->new->utf8->space_after (1); + my $yaml = $to_yaml->encode ($ref) . "\n"; + +This will I<usually> generate JSON texts that also parse as valid +YAML. Please note that YAML has hardcoded limits on (simple) object key +lengths that JSON doesn't have and also has different and incompatible +unicode character escape syntax, so you should make sure that your hash +keys are noticeably shorter than the 1024 "stream characters" YAML allows +and that you do not have characters with codepoint values outside the +Unicode BMP (basic multilingual page). YAML also does not allow C<\/> +sequences in strings (which JSON::XS does not I<currently> generate, but +other JSON generators might). + +There might be other incompatibilities that I am not aware of (or the YAML +specification has been changed yet again - it does so quite often). In +general you should not try to generate YAML with a JSON generator or vice +versa, or try to parse JSON with a YAML parser or vice versa: chances are +high that you will run into severe interoperability problems when you +least expect it. + +=over 4 + +=item (*) + +I have been pressured multiple times by Brian Ingerson (one of the +authors of the YAML specification) to remove this paragraph, despite him +acknowledging that the actual incompatibilities exist. As I was personally +bitten by this "JSON is YAML" lie, I refused and said I will continue to +educate people about these issues, so others do not run into the same +problem again and again. After this, Brian called me a (quote)I<complete +and worthless idiot>(unquote). + +In my opinion, instead of pressuring and insulting people who actually +clarify issues with YAML and the wrong statements of some of its +proponents, I would kindly suggest reading the JSON spec (which is not +that difficult or long) and finally make YAML compatible to it, and +educating users about the changes, instead of spreading lies about the +real compatibility for many I<years> and trying to silence people who +point out that it isn't true. + +Addendum/2009: the YAML 1.2 spec is still incompatible with JSON, even +though the incompatibilities have been documented (and are known to Brian) +for many years and the spec makes explicit claims that YAML is a superset +of JSON. It would be so easy to fix, but apparently, bullying people and +corrupting userdata is so much easier. + +=back + + +=head2 SPEED + +It seems that JSON::XS is surprisingly fast, as shown in the following +tables. They have been generated with the help of the C<eg/bench> program +in the JSON::XS distribution, to make it easy to compare on your own +system. + +First comes a comparison between various modules using +a very short single-line JSON string (also available at +L<http://dist.schmorp.de/misc/json/short.json>). + + {"method": "handleMessage", "params": ["user1", + "we were just talking"], "id": null, "array":[1,11,234,-5,1e5,1e7, + 1, 0]} + +It shows the number of encodes/decodes per second (JSON::XS uses +the functional interface, while JSON::XS/2 uses the OO interface +with pretty-printing and hashkey sorting enabled, JSON::XS/3 enables +shrink. JSON::DWIW/DS uses the deserialise function, while JSON::DWIW::FJ +uses the from_json method). Higher is better: + + module | encode | decode | + --------------|------------|------------| + JSON::DWIW/DS | 86302.551 | 102300.098 | + JSON::DWIW/FJ | 86302.551 | 75983.768 | + JSON::PP | 15827.562 | 6638.658 | + JSON::Syck | 63358.066 | 47662.545 | + JSON::XS | 511500.488 | 511500.488 | + JSON::XS/2 | 291271.111 | 388361.481 | + JSON::XS/3 | 361577.931 | 361577.931 | + Storable | 66788.280 | 265462.278 | + --------------+------------+------------+ + +That is, JSON::XS is almost six times faster than JSON::DWIW on encoding, +about five times faster on decoding, and over thirty to seventy times +faster than JSON's pure perl implementation. It also compares favourably +to Storable for small amounts of data. + +Using a longer test string (roughly 18KB, generated from Yahoo! Locals +search API (L<http://dist.schmorp.de/misc/json/long.json>). + + module | encode | decode | + --------------|------------|------------| + JSON::DWIW/DS | 1647.927 | 2673.916 | + JSON::DWIW/FJ | 1630.249 | 2596.128 | + JSON::PP | 400.640 | 62.311 | + JSON::Syck | 1481.040 | 1524.869 | + JSON::XS | 20661.596 | 9541.183 | + JSON::XS/2 | 10683.403 | 9416.938 | + JSON::XS/3 | 20661.596 | 9400.054 | + Storable | 19765.806 | 10000.725 | + --------------+------------+------------+ + +Again, JSON::XS leads by far (except for Storable which non-surprisingly +decodes a bit faster). + +On large strings containing lots of high Unicode characters, some modules +(such as JSON::PC) seem to decode faster than JSON::XS, but the result +will be broken due to missing (or wrong) Unicode handling. Others refuse +to decode or encode properly, so it was impossible to prepare a fair +comparison table for that case. + + +=head1 SECURITY CONSIDERATIONS + +When you are using JSON in a protocol, talking to untrusted potentially +hostile creatures requires relatively few measures. + +First of all, your JSON decoder should be secure, that is, should not have +any buffer overflows. Obviously, this module should ensure that and I am +trying hard on making that true, but you never know. + +Second, you need to avoid resource-starving attacks. That means you should +limit the size of JSON texts you accept, or make sure then when your +resources run out, that's just fine (e.g. by using a separate process that +can crash safely). The size of a JSON text in octets or characters is +usually a good indication of the size of the resources required to decode +it into a Perl structure. While JSON::XS can check the size of the JSON +text, it might be too late when you already have it in memory, so you +might want to check the size before you accept the string. + +Third, JSON::XS recurses using the C stack when decoding objects and +arrays. The C stack is a limited resource: for instance, on my amd64 +machine with 8MB of stack size I can decode around 180k nested arrays but +only 14k nested JSON objects (due to perl itself recursing deeply on croak +to free the temporary). If that is exceeded, the program crashes. To be +conservative, the default nesting limit is set to 512. If your process +has a smaller stack, you should adjust this setting accordingly with the +C<max_depth> method. + +Something else could bomb you, too, that I forgot to think of. In that +case, you get to keep the pieces. I am always open for hints, though... + +Also keep in mind that JSON::XS might leak contents of your Perl data +structures in its error messages, so when you serialise sensitive +information you might want to make sure that exceptions thrown by JSON::XS +will not end up in front of untrusted eyes. + +If you are using JSON::XS to return packets to consumption +by JavaScript scripts in a browser you should have a look at +L<http://blog.archive.jpsykes.com/47/practical-csrf-and-json-security/> to +see whether you are vulnerable to some common attack vectors (which really +are browser design bugs, but it is still you who will have to deal with +it, as major browser developers care only for features, not about getting +security right). + + +=head1 "OLD" VS. "NEW" JSON (RFC 4627 VS. RFC 7159) + +TL;DR: Due to security concerns, JSON::XS will not allow scalar data in +JSON texts by default - you need to create your own JSON::XS object and +enable C<allow_nonref>: + + + my $json = JSON::XS->new->allow_nonref; + + $text = $json->encode ($data); + $data = $json->decode ($text); + +The long version: JSON being an important and supposedly stable format, +the IETF standardised it as RFC 4627 in 2006. Unfortunately, the inventor +of JSON, Dougles Crockford, unilaterally changed the definition of JSON in +javascript. Rather than create a fork, the IETF decided to standardise the +new syntax (apparently, so Iw as told, without finding it very amusing). + +The biggest difference between thed original JSON and the new JSON is that +the new JSON supports scalars (anything other than arrays and objects) at +the toplevel of a JSON text. While this is strictly backwards compatible +to older versions, it breaks a number of protocols that relied on sending +JSON back-to-back, and is a minor security concern. + +For example, imagine you have two banks communicating, and on one side, +trhe JSON coder gets upgraded. Two messages, such as C<10> and C<1000> +might then be confused to mean C<101000>, something that couldn't happen +in the original JSON, because niether of these messages would be valid +JSON. + +If one side accepts these messages, then an upgrade in the coder on either +side could result in this becoming exploitable. + +This module has always allowed these messages as an optional extension, by +default disabled. The security concerns are the reason why the default is +still disabled, but future versions might/will likely upgrade to the newer +RFC as default format, so you are advised to check your implementation +and/or override the default with C<< ->allow_nonref (0) >> to ensure that +future versions are safe. + + +=head1 INTEROPERABILITY WITH OTHER MODULES + +C<JSON::XS> uses the L<Types::Serialiser> module to provide boolean +constants. That means that the JSON true and false values will be +comaptible to true and false values of other modules that do the same, +such as L<JSON::PP> and L<CBOR::XS>. + + +=head1 INTEROPERABILITY WITH OTHER JSON DECODERS + +As long as you only serialise data that can be directly expressed in JSON, +C<JSON::XS> is incapable of generating invalid JSON output (modulo bugs, +but C<JSON::XS> has found more bugs in the official JSON testsuite (1) +than the official JSON testsuite has found in C<JSON::XS> (0)). + +When you have trouble decoding JSON generated by this module using other +decoders, then it is very likely that you have an encoding mismatch or the +other decoder is broken. + +When decoding, C<JSON::XS> is strict by default and will likely catch all +errors. There are currently two settings that change this: C<relaxed> +makes C<JSON::XS> accept (but not generate) some non-standard extensions, +and C<allow_tags> will allow you to encode and decode Perl objects, at the +cost of not outputting valid JSON anymore. + +=head2 TAGGED VALUE SYNTAX AND STANDARD JSON EN/DECODERS + +When you use C<allow_tags> to use the extended (and also nonstandard and +invalid) JSON syntax for serialised objects, and you still want to decode +the generated When you want to serialise objects, you can run a regex +to replace the tagged syntax by standard JSON arrays (it only works for +"normal" package names without comma, newlines or single colons). First, +the readable Perl version: + + # if your FREEZE methods return no values, you need this replace first: + $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[\s*\]/[$1]/gx; + + # this works for non-empty constructor arg lists: + $json =~ s/\( \s* (" (?: [^\\":,]+|\\.|::)* ") \s* \) \s* \[/[$1,/gx; + +And here is a less readable version that is easy to adapt to other +languages: + + $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/[$1,/g; + +Here is an ECMAScript version (same regex): + + json = json.replace (/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/g, "[$1,"); + +Since this syntax converts to standard JSON arrays, it might be hard to +distinguish serialised objects from normal arrays. You can prepend a +"magic number" as first array element to reduce chances of a collision: + + $json =~ s/\(\s*("([^\\":,]+|\\.|::)*")\s*\)\s*\[/["XU1peReLzT4ggEllLanBYq4G9VzliwKF",$1,/g; + +And after decoding the JSON text, you could walk the data +structure looking for arrays with a first element of +C<XU1peReLzT4ggEllLanBYq4G9VzliwKF>. + +The same approach can be used to create the tagged format with another +encoder. First, you create an array with the magic string as first member, +the classname as second, and constructor arguments last, encode it as part +of your JSON structure, and then: + + $json =~ s/\[\s*"XU1peReLzT4ggEllLanBYq4G9VzliwKF"\s*,\s*("([^\\":,]+|\\.|::)*")\s*,/($1)[/g; + +Again, this has some limitations - the magic string must not be encoded +with character escapes, and the constructor arguments must be non-empty. + + +=head1 RFC7159 + +Since this module was written, Google has written a new JSON RFC, RFC 7159 +(and RFC7158). Unfortunately, this RFC breaks compatibility with both the +original JSON specification on www.json.org and RFC4627. + +As far as I can see, you can get partial compatibility when parsing by +using C<< ->allow_nonref >>. However, consider the security implications +of doing so. + +I haven't decided yet when to break compatibility with RFC4627 by default +(and potentially leave applications insecure) and change the default to +follow RFC7159, but application authors are well advised to call C<< +->allow_nonref(0) >> even if this is the current default, if they cannot +handle non-reference values, in preparation for the day when the default +will change. + + +=head1 (I-)THREADS + +This module is I<not> guaranteed to be ithread (or MULTIPLICITY-) safe +and there are no plans to change this. Note that perl's builtin so-called +theeads/ithreads are officially deprecated and should not be used. + + +=head1 THE PERILS OF SETLOCALE + +Sometimes people avoid the Perl locale support and directly call the +system's setlocale function with C<LC_ALL>. + +This breaks both perl and modules such as JSON::XS, as stringification of +numbers no longer works correctly (e.g. C<$x = 0.1; print "$x"+1> might +print C<1>, and JSON::XS might output illegal JSON as JSON::XS relies on +perl to stringify numbers). + +The solution is simple: don't call C<setlocale>, or use it for only those +categories you need, such as C<LC_MESSAGES> or C<LC_CTYPE>. + +If you need C<LC_NUMERIC>, you should enable it only around the code that +actually needs it (avoiding stringification of numbers), and restore it +afterwards. + + +=head1 BUGS + +While the goal of this module is to be correct, that unfortunately does +not mean it's bug-free, only that I think its design is bug-free. If you +keep reporting bugs they will be fixed swiftly, though. + +Please refrain from using rt.cpan.org or any other bug reporting +service. I put the contact address into my modules for a reason. + +=cut + +BEGIN { + *true = \$Types::Serialiser::true; + *true = \&Types::Serialiser::true; + *false = \$Types::Serialiser::false; + *false = \&Types::Serialiser::false; + *is_bool = \&Types::Serialiser::is_bool; + + *JSON::XS::Boolean:: = *Types::Serialiser::Boolean::; +} + +XSLoader::load "JSON::XS", $VERSION; + +=head1 SEE ALSO + +The F<json_xs> command line utility for quick experiments. + +=head1 AUTHOR + + Marc Lehmann <schmorp@schmorp.de> + http://home.schmorp.de/ + +=cut + +1 + diff --git a/Master/tlpkg/tlperl/site/lib/JSON/XS/Boolean.pm b/Master/tlpkg/tlperl/site/lib/JSON/XS/Boolean.pm new file mode 100644 index 00000000000..67f704b2a72 --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/JSON/XS/Boolean.pm @@ -0,0 +1,31 @@ +=head1 NAME + +JSON::XS::Boolean - dummy module providing JSON::XS::Boolean + +=head1 SYNOPSIS + + # do not "use" yourself + +=head1 DESCRIPTION + +This module exists only to provide overload resolution for Storable and +similar modules. It's only needed for compatibility with data serialised +(by other modules such as Storable) that was decoded by JSON::XS versions +before 3.0. + +Since 3.0, JSON::PP::Boolean has replaced it. Support for +JSON::XS::Boolean will be removed in a future release. + +=cut + +use JSON::XS (); + +1; + +=head1 AUTHOR + + Marc Lehmann <schmorp@schmorp.de> + http://home.schmorp.de/ + +=cut + diff --git a/Master/tlpkg/tlperl/site/lib/JSON/backportPP.pm b/Master/tlpkg/tlperl/site/lib/JSON/backportPP.pm new file mode 100644 index 00000000000..36132a3307b --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/JSON/backportPP.pm @@ -0,0 +1,2854 @@ +package # This is JSON::backportPP + JSON::PP; + +# JSON-2.0 + +use 5.005; +use strict; + +use Exporter (); +BEGIN { @JSON::backportPP::ISA = ('Exporter') } + +use overload (); +use JSON::backportPP::Boolean; + +use Carp (); +#use Devel::Peek; + +$JSON::backportPP::VERSION = '2.94'; + +@JSON::PP::EXPORT = qw(encode_json decode_json from_json to_json); + +# instead of hash-access, i tried index-access for speed. +# but this method is not faster than what i expected. so it will be changed. + +use constant P_ASCII => 0; +use constant P_LATIN1 => 1; +use constant P_UTF8 => 2; +use constant P_INDENT => 3; +use constant P_CANONICAL => 4; +use constant P_SPACE_BEFORE => 5; +use constant P_SPACE_AFTER => 6; +use constant P_ALLOW_NONREF => 7; +use constant P_SHRINK => 8; +use constant P_ALLOW_BLESSED => 9; +use constant P_CONVERT_BLESSED => 10; +use constant P_RELAXED => 11; + +use constant P_LOOSE => 12; +use constant P_ALLOW_BIGNUM => 13; +use constant P_ALLOW_BAREKEY => 14; +use constant P_ALLOW_SINGLEQUOTE => 15; +use constant P_ESCAPE_SLASH => 16; +use constant P_AS_NONBLESSED => 17; + +use constant P_ALLOW_UNKNOWN => 18; + +use constant OLD_PERL => $] < 5.008 ? 1 : 0; +use constant USE_B => 0; + +BEGIN { +if (USE_B) { + require B; +} +} + +BEGIN { + my @xs_compati_bit_properties = qw( + latin1 ascii utf8 indent canonical space_before space_after allow_nonref shrink + allow_blessed convert_blessed relaxed allow_unknown + ); + my @pp_bit_properties = qw( + allow_singlequote allow_bignum loose + allow_barekey escape_slash as_nonblessed + ); + + # Perl version check, Unicode handling is enabled? + # Helper module sets @JSON::PP::_properties. + if ( OLD_PERL ) { + my $helper = $] >= 5.006 ? 'JSON::backportPP::Compat5006' : 'JSON::backportPP::Compat5005'; + eval qq| require $helper |; + if ($@) { Carp::croak $@; } + } + + for my $name (@xs_compati_bit_properties, @pp_bit_properties) { + my $property_id = 'P_' . uc($name); + + eval qq/ + sub $name { + my \$enable = defined \$_[1] ? \$_[1] : 1; + + if (\$enable) { + \$_[0]->{PROPS}->[$property_id] = 1; + } + else { + \$_[0]->{PROPS}->[$property_id] = 0; + } + + \$_[0]; + } + + sub get_$name { + \$_[0]->{PROPS}->[$property_id] ? 1 : ''; + } + /; + } + +} + + + +# Functions + +my $JSON; # cache + +sub encode_json ($) { # encode + ($JSON ||= __PACKAGE__->new->utf8)->encode(@_); +} + + +sub decode_json { # decode + ($JSON ||= __PACKAGE__->new->utf8)->decode(@_); +} + +# Obsoleted + +sub to_json($) { + Carp::croak ("JSON::PP::to_json has been renamed to encode_json."); +} + + +sub from_json($) { + Carp::croak ("JSON::PP::from_json has been renamed to decode_json."); +} + + +# Methods + +sub new { + my $class = shift; + my $self = { + max_depth => 512, + max_size => 0, + indent_length => 3, + }; + + bless $self, $class; +} + + +sub encode { + return $_[0]->PP_encode_json($_[1]); +} + + +sub decode { + return $_[0]->PP_decode_json($_[1], 0x00000000); +} + + +sub decode_prefix { + return $_[0]->PP_decode_json($_[1], 0x00000001); +} + + +# accessor + + +# pretty printing + +sub pretty { + my ($self, $v) = @_; + my $enable = defined $v ? $v : 1; + + if ($enable) { # indent_length(3) for JSON::XS compatibility + $self->indent(1)->space_before(1)->space_after(1); + } + else { + $self->indent(0)->space_before(0)->space_after(0); + } + + $self; +} + +# etc + +sub max_depth { + my $max = defined $_[1] ? $_[1] : 0x80000000; + $_[0]->{max_depth} = $max; + $_[0]; +} + + +sub get_max_depth { $_[0]->{max_depth}; } + + +sub max_size { + my $max = defined $_[1] ? $_[1] : 0; + $_[0]->{max_size} = $max; + $_[0]; +} + + +sub get_max_size { $_[0]->{max_size}; } + + +sub filter_json_object { + if (defined $_[1] and ref $_[1] eq 'CODE') { + $_[0]->{cb_object} = $_[1]; + } else { + delete $_[0]->{cb_object}; + } + $_[0]->{F_HOOK} = ($_[0]->{cb_object} or $_[0]->{cb_sk_object}) ? 1 : 0; + $_[0]; +} + +sub filter_json_single_key_object { + if (@_ == 1 or @_ > 3) { + Carp::croak("Usage: JSON::PP::filter_json_single_key_object(self, key, callback = undef)"); + } + if (defined $_[2] and ref $_[2] eq 'CODE') { + $_[0]->{cb_sk_object}->{$_[1]} = $_[2]; + } else { + delete $_[0]->{cb_sk_object}->{$_[1]}; + delete $_[0]->{cb_sk_object} unless %{$_[0]->{cb_sk_object} || {}}; + } + $_[0]->{F_HOOK} = ($_[0]->{cb_object} or $_[0]->{cb_sk_object}) ? 1 : 0; + $_[0]; +} + +sub indent_length { + if (!defined $_[1] or $_[1] > 15 or $_[1] < 0) { + Carp::carp "The acceptable range of indent_length() is 0 to 15."; + } + else { + $_[0]->{indent_length} = $_[1]; + } + $_[0]; +} + +sub get_indent_length { + $_[0]->{indent_length}; +} + +sub sort_by { + $_[0]->{sort_by} = defined $_[1] ? $_[1] : 1; + $_[0]; +} + +sub allow_bigint { + Carp::carp("allow_bigint() is obsoleted. use allow_bignum() instead."); + $_[0]->allow_bignum; +} + +############################### + +### +### Perl => JSON +### + + +{ # Convert + + my $max_depth; + my $indent; + my $ascii; + my $latin1; + my $utf8; + my $space_before; + my $space_after; + my $canonical; + my $allow_blessed; + my $convert_blessed; + + my $indent_length; + my $escape_slash; + my $bignum; + my $as_nonblessed; + + my $depth; + my $indent_count; + my $keysort; + + + sub PP_encode_json { + my $self = shift; + my $obj = shift; + + $indent_count = 0; + $depth = 0; + + my $props = $self->{PROPS}; + + ($ascii, $latin1, $utf8, $indent, $canonical, $space_before, $space_after, $allow_blessed, + $convert_blessed, $escape_slash, $bignum, $as_nonblessed) + = @{$props}[P_ASCII .. P_SPACE_AFTER, P_ALLOW_BLESSED, P_CONVERT_BLESSED, + P_ESCAPE_SLASH, P_ALLOW_BIGNUM, P_AS_NONBLESSED]; + + ($max_depth, $indent_length) = @{$self}{qw/max_depth indent_length/}; + + $keysort = $canonical ? sub { $a cmp $b } : undef; + + if ($self->{sort_by}) { + $keysort = ref($self->{sort_by}) eq 'CODE' ? $self->{sort_by} + : $self->{sort_by} =~ /\D+/ ? $self->{sort_by} + : sub { $a cmp $b }; + } + + encode_error("hash- or arrayref expected (not a simple scalar, use allow_nonref to allow this)") + if(!ref $obj and !$props->[ P_ALLOW_NONREF ]); + + my $str = $self->object_to_json($obj); + + $str .= "\n" if ( $indent ); # JSON::XS 2.26 compatible + + unless ($ascii or $latin1 or $utf8) { + utf8::upgrade($str); + } + + if ($props->[ P_SHRINK ]) { + utf8::downgrade($str, 1); + } + + return $str; + } + + + sub object_to_json { + my ($self, $obj) = @_; + my $type = ref($obj); + + if($type eq 'HASH'){ + return $self->hash_to_json($obj); + } + elsif($type eq 'ARRAY'){ + return $self->array_to_json($obj); + } + elsif ($type) { # blessed object? + if (blessed($obj)) { + + return $self->value_to_json($obj) if ( $obj->isa('JSON::PP::Boolean') ); + + if ( $convert_blessed and $obj->can('TO_JSON') ) { + my $result = $obj->TO_JSON(); + if ( defined $result and ref( $result ) ) { + if ( refaddr( $obj ) eq refaddr( $result ) ) { + encode_error( sprintf( + "%s::TO_JSON method returned same object as was passed instead of a new one", + ref $obj + ) ); + } + } + + return $self->object_to_json( $result ); + } + + return "$obj" if ( $bignum and _is_bignum($obj) ); + + if ($allow_blessed) { + return $self->blessed_to_json($obj) if ($as_nonblessed); # will be removed. + return 'null'; + } + encode_error( sprintf("encountered object '%s', but neither allow_blessed " + . "nor convert_blessed settings are enabled", $obj) + ); + } + else { + return $self->value_to_json($obj); + } + } + else{ + return $self->value_to_json($obj); + } + } + + + sub hash_to_json { + my ($self, $obj) = @_; + my @res; + + encode_error("json text or perl structure exceeds maximum nesting level (max_depth set too low?)") + if (++$depth > $max_depth); + + my ($pre, $post) = $indent ? $self->_up_indent() : ('', ''); + my $del = ($space_before ? ' ' : '') . ':' . ($space_after ? ' ' : ''); + + for my $k ( _sort( $obj ) ) { + if ( OLD_PERL ) { utf8::decode($k) } # key for Perl 5.6 / be optimized + push @res, $self->string_to_json( $k ) + . $del + . ( ref $obj->{$k} ? $self->object_to_json( $obj->{$k} ) : $self->value_to_json( $obj->{$k} ) ); + } + + --$depth; + $self->_down_indent() if ($indent); + + return '{}' unless @res; + return '{' . $pre . join( ",$pre", @res ) . $post . '}'; + } + + + sub array_to_json { + my ($self, $obj) = @_; + my @res; + + encode_error("json text or perl structure exceeds maximum nesting level (max_depth set too low?)") + if (++$depth > $max_depth); + + my ($pre, $post) = $indent ? $self->_up_indent() : ('', ''); + + for my $v (@$obj){ + push @res, ref($v) ? $self->object_to_json($v) : $self->value_to_json($v); + } + + --$depth; + $self->_down_indent() if ($indent); + + return '[]' unless @res; + return '[' . $pre . join( ",$pre", @res ) . $post . ']'; + } + + sub _looks_like_number { + my $value = shift; + if (USE_B) { + my $b_obj = B::svref_2object(\$value); + my $flags = $b_obj->FLAGS; + return 1 if $flags & ( B::SVp_IOK() | B::SVp_NOK() ) and !( $flags & B::SVp_POK() ); + return; + } else { + no warnings 'numeric'; + # detect numbers + # string & "" -> "" + # number & "" -> 0 (with warning) + # nan and inf can detect as numbers, so check with * 0 + return unless length((my $dummy = "") & $value); + return unless 0 + $value eq $value; + return 1 if $value * 0 == 0; + return -1; # inf/nan + } + } + + sub value_to_json { + my ($self, $value) = @_; + + return 'null' if(!defined $value); + + my $type = ref($value); + + if (!$type) { + if (_looks_like_number($value)) { + return $value; + } + return $self->string_to_json($value); + } + elsif( blessed($value) and $value->isa('JSON::PP::Boolean') ){ + return $$value == 1 ? 'true' : 'false'; + } + else { + if ((overload::StrVal($value) =~ /=(\w+)/)[0]) { + return $self->value_to_json("$value"); + } + + if ($type eq 'SCALAR' and defined $$value) { + return $$value eq '1' ? 'true' + : $$value eq '0' ? 'false' + : $self->{PROPS}->[ P_ALLOW_UNKNOWN ] ? 'null' + : encode_error("cannot encode reference to scalar"); + } + + if ( $self->{PROPS}->[ P_ALLOW_UNKNOWN ] ) { + return 'null'; + } + else { + if ( $type eq 'SCALAR' or $type eq 'REF' ) { + encode_error("cannot encode reference to scalar"); + } + else { + encode_error("encountered $value, but JSON can only represent references to arrays or hashes"); + } + } + + } + } + + + my %esc = ( + "\n" => '\n', + "\r" => '\r', + "\t" => '\t', + "\f" => '\f', + "\b" => '\b', + "\"" => '\"', + "\\" => '\\\\', + "\'" => '\\\'', + ); + + + sub string_to_json { + my ($self, $arg) = @_; + + $arg =~ s/([\x22\x5c\n\r\t\f\b])/$esc{$1}/g; + $arg =~ s/\//\\\//g if ($escape_slash); + $arg =~ s/([\x00-\x08\x0b\x0e-\x1f])/'\\u00' . unpack('H2', $1)/eg; + + if ($ascii) { + $arg = JSON_PP_encode_ascii($arg); + } + + if ($latin1) { + $arg = JSON_PP_encode_latin1($arg); + } + + if ($utf8) { + utf8::encode($arg); + } + + return '"' . $arg . '"'; + } + + + sub blessed_to_json { + my $reftype = reftype($_[1]) || ''; + if ($reftype eq 'HASH') { + return $_[0]->hash_to_json($_[1]); + } + elsif ($reftype eq 'ARRAY') { + return $_[0]->array_to_json($_[1]); + } + else { + return 'null'; + } + } + + + sub encode_error { + my $error = shift; + Carp::croak "$error"; + } + + + sub _sort { + defined $keysort ? (sort $keysort (keys %{$_[0]})) : keys %{$_[0]}; + } + + + sub _up_indent { + my $self = shift; + my $space = ' ' x $indent_length; + + my ($pre,$post) = ('',''); + + $post = "\n" . $space x $indent_count; + + $indent_count++; + + $pre = "\n" . $space x $indent_count; + + return ($pre,$post); + } + + + sub _down_indent { $indent_count--; } + + + sub PP_encode_box { + { + depth => $depth, + indent_count => $indent_count, + }; + } + +} # Convert + + +sub _encode_ascii { + join('', + map { + $_ <= 127 ? + chr($_) : + $_ <= 65535 ? + sprintf('\u%04x', $_) : sprintf('\u%x\u%x', _encode_surrogates($_)); + } unpack('U*', $_[0]) + ); +} + + +sub _encode_latin1 { + join('', + map { + $_ <= 255 ? + chr($_) : + $_ <= 65535 ? + sprintf('\u%04x', $_) : sprintf('\u%x\u%x', _encode_surrogates($_)); + } unpack('U*', $_[0]) + ); +} + + +sub _encode_surrogates { # from perlunicode + my $uni = $_[0] - 0x10000; + return ($uni / 0x400 + 0xD800, $uni % 0x400 + 0xDC00); +} + + +sub _is_bignum { + $_[0]->isa('Math::BigInt') or $_[0]->isa('Math::BigFloat'); +} + + + +# +# JSON => Perl +# + +my $max_intsize; + +BEGIN { + my $checkint = 1111; + for my $d (5..64) { + $checkint .= 1; + my $int = eval qq| $checkint |; + if ($int =~ /[eE]/) { + $max_intsize = $d - 1; + last; + } + } +} + +{ # PARSE + + my %escapes = ( # by Jeremy Muhlich <jmuhlich [at] bitflood.org> + b => "\x8", + t => "\x9", + n => "\xA", + f => "\xC", + r => "\xD", + '\\' => '\\', + '"' => '"', + '/' => '/', + ); + + my $text; # json data + my $at; # offset + my $ch; # first character + my $len; # text length (changed according to UTF8 or NON UTF8) + # INTERNAL + my $depth; # nest counter + my $encoding; # json text encoding + my $is_valid_utf8; # temp variable + my $utf8_len; # utf8 byte length + # FLAGS + my $utf8; # must be utf8 + my $max_depth; # max nest number of objects and arrays + my $max_size; + my $relaxed; + my $cb_object; + my $cb_sk_object; + + my $F_HOOK; + + my $allow_bignum; # using Math::BigInt/BigFloat + my $singlequote; # loosely quoting + my $loose; # + my $allow_barekey; # bareKey + + sub _detect_utf_encoding { + my $text = shift; + my @octets = unpack('C4', $text); + return 'unknown' unless defined $octets[3]; + return ( $octets[0] and $octets[1]) ? 'UTF-8' + : (!$octets[0] and $octets[1]) ? 'UTF-16BE' + : (!$octets[0] and !$octets[1]) ? 'UTF-32BE' + : ( $octets[2] ) ? 'UTF-16LE' + : (!$octets[2] ) ? 'UTF-32LE' + : 'unknown'; + } + + sub PP_decode_json { + my ($self, $want_offset); + + ($self, $text, $want_offset) = @_; + + ($at, $ch, $depth) = (0, '', 0); + + if ( !defined $text or ref $text ) { + decode_error("malformed JSON string, neither array, object, number, string or atom"); + } + + my $props = $self->{PROPS}; + + ($utf8, $relaxed, $loose, $allow_bignum, $allow_barekey, $singlequote) + = @{$props}[P_UTF8, P_RELAXED, P_LOOSE .. P_ALLOW_SINGLEQUOTE]; + + if ( $utf8 ) { + $encoding = _detect_utf_encoding($text); + if ($encoding ne 'UTF-8' and $encoding ne 'unknown') { + require Encode; + Encode::from_to($text, $encoding, 'utf-8'); + } else { + utf8::downgrade( $text, 1 ) or Carp::croak("Wide character in subroutine entry"); + } + } + else { + utf8::upgrade( $text ); + utf8::encode( $text ); + } + + $len = length $text; + + ($max_depth, $max_size, $cb_object, $cb_sk_object, $F_HOOK) + = @{$self}{qw/max_depth max_size cb_object cb_sk_object F_HOOK/}; + + if ($max_size > 1) { + use bytes; + my $bytes = length $text; + decode_error( + sprintf("attempted decode of JSON text of %s bytes size, but max_size is set to %s" + , $bytes, $max_size), 1 + ) if ($bytes > $max_size); + } + + white(); # remove head white space + + decode_error("malformed JSON string, neither array, object, number, string or atom") unless defined $ch; # Is there a first character for JSON structure? + + my $result = value(); + + if ( !$props->[ P_ALLOW_NONREF ] and !ref $result ) { + decode_error( + 'JSON text must be an object or array (but found number, string, true, false or null,' + . ' use allow_nonref to allow this)', 1); + } + + Carp::croak('something wrong.') if $len < $at; # we won't arrive here. + + my $consumed = defined $ch ? $at - 1 : $at; # consumed JSON text length + + white(); # remove tail white space + + return ( $result, $consumed ) if $want_offset; # all right if decode_prefix + + decode_error("garbage after JSON object") if defined $ch; + + $result; + } + + + sub next_chr { + return $ch = undef if($at >= $len); + $ch = substr($text, $at++, 1); + } + + + sub value { + white(); + return if(!defined $ch); + return object() if($ch eq '{'); + return array() if($ch eq '['); + return string() if($ch eq '"' or ($singlequote and $ch eq "'")); + return number() if($ch =~ /[0-9]/ or $ch eq '-'); + return word(); + } + + sub string { + my $utf16; + my $is_utf8; + + ($is_valid_utf8, $utf8_len) = ('', 0); + + my $s = ''; # basically UTF8 flag on + + if($ch eq '"' or ($singlequote and $ch eq "'")){ + my $boundChar = $ch; + + OUTER: while( defined(next_chr()) ){ + + if($ch eq $boundChar){ + next_chr(); + + if ($utf16) { + decode_error("missing low surrogate character in surrogate pair"); + } + + utf8::decode($s) if($is_utf8); + + return $s; + } + elsif($ch eq '\\'){ + next_chr(); + if(exists $escapes{$ch}){ + $s .= $escapes{$ch}; + } + elsif($ch eq 'u'){ # UNICODE handling + my $u = ''; + + for(1..4){ + $ch = next_chr(); + last OUTER if($ch !~ /[0-9a-fA-F]/); + $u .= $ch; + } + + # U+D800 - U+DBFF + if ($u =~ /^[dD][89abAB][0-9a-fA-F]{2}/) { # UTF-16 high surrogate? + $utf16 = $u; + } + # U+DC00 - U+DFFF + elsif ($u =~ /^[dD][c-fC-F][0-9a-fA-F]{2}/) { # UTF-16 low surrogate? + unless (defined $utf16) { + decode_error("missing high surrogate character in surrogate pair"); + } + $is_utf8 = 1; + $s .= JSON_PP_decode_surrogates($utf16, $u) || next; + $utf16 = undef; + } + else { + if (defined $utf16) { + decode_error("surrogate pair expected"); + } + + if ( ( my $hex = hex( $u ) ) > 127 ) { + $is_utf8 = 1; + $s .= JSON_PP_decode_unicode($u) || next; + } + else { + $s .= chr $hex; + } + } + + } + else{ + unless ($loose) { + $at -= 2; + decode_error('illegal backslash escape sequence in string'); + } + $s .= $ch; + } + } + else{ + + if ( ord $ch > 127 ) { + unless( $ch = is_valid_utf8($ch) ) { + $at -= 1; + decode_error("malformed UTF-8 character in JSON string"); + } + else { + $at += $utf8_len - 1; + } + + $is_utf8 = 1; + } + + if (!$loose) { + if ($ch =~ /[\x00-\x1f\x22\x5c]/) { # '/' ok + $at--; + decode_error('invalid character encountered while parsing JSON string'); + } + } + + $s .= $ch; + } + } + } + + decode_error("unexpected end of string while parsing JSON string"); + } + + + sub white { + while( defined $ch ){ + if($ch eq '' or $ch =~ /\A[ \t\r\n]\z/){ + next_chr(); + } + elsif($relaxed and $ch eq '/'){ + next_chr(); + if(defined $ch and $ch eq '/'){ + 1 while(defined(next_chr()) and $ch ne "\n" and $ch ne "\r"); + } + elsif(defined $ch and $ch eq '*'){ + next_chr(); + while(1){ + if(defined $ch){ + if($ch eq '*'){ + if(defined(next_chr()) and $ch eq '/'){ + next_chr(); + last; + } + } + else{ + next_chr(); + } + } + else{ + decode_error("Unterminated comment"); + } + } + next; + } + else{ + $at--; + decode_error("malformed JSON string, neither array, object, number, string or atom"); + } + } + else{ + if ($relaxed and $ch eq '#') { # correctly? + pos($text) = $at; + $text =~ /\G([^\n]*(?:\r\n|\r|\n|$))/g; + $at = pos($text); + next_chr; + next; + } + + last; + } + } + } + + + sub array { + my $a = $_[0] || []; # you can use this code to use another array ref object. + + decode_error('json text or perl structure exceeds maximum nesting level (max_depth set too low?)') + if (++$depth > $max_depth); + + next_chr(); + white(); + + if(defined $ch and $ch eq ']'){ + --$depth; + next_chr(); + return $a; + } + else { + while(defined($ch)){ + push @$a, value(); + + white(); + + if (!defined $ch) { + last; + } + + if($ch eq ']'){ + --$depth; + next_chr(); + return $a; + } + + if($ch ne ','){ + last; + } + + next_chr(); + white(); + + if ($relaxed and $ch eq ']') { + --$depth; + next_chr(); + return $a; + } + + } + } + + $at-- if defined $ch and $ch ne ''; + decode_error(", or ] expected while parsing array"); + } + + + sub object { + my $o = $_[0] || {}; # you can use this code to use another hash ref object. + my $k; + + decode_error('json text or perl structure exceeds maximum nesting level (max_depth set too low?)') + if (++$depth > $max_depth); + next_chr(); + white(); + + if(defined $ch and $ch eq '}'){ + --$depth; + next_chr(); + if ($F_HOOK) { + return _json_object_hook($o); + } + return $o; + } + else { + while (defined $ch) { + $k = ($allow_barekey and $ch ne '"' and $ch ne "'") ? bareKey() : string(); + white(); + + if(!defined $ch or $ch ne ':'){ + $at--; + decode_error("':' expected"); + } + + next_chr(); + $o->{$k} = value(); + white(); + + last if (!defined $ch); + + if($ch eq '}'){ + --$depth; + next_chr(); + if ($F_HOOK) { + return _json_object_hook($o); + } + return $o; + } + + if($ch ne ','){ + last; + } + + next_chr(); + white(); + + if ($relaxed and $ch eq '}') { + --$depth; + next_chr(); + if ($F_HOOK) { + return _json_object_hook($o); + } + return $o; + } + + } + + } + + $at-- if defined $ch and $ch ne ''; + decode_error(", or } expected while parsing object/hash"); + } + + + sub bareKey { # doesn't strictly follow Standard ECMA-262 3rd Edition + my $key; + while($ch =~ /[^\x00-\x23\x25-\x2F\x3A-\x40\x5B-\x5E\x60\x7B-\x7F]/){ + $key .= $ch; + next_chr(); + } + return $key; + } + + + sub word { + my $word = substr($text,$at-1,4); + + if($word eq 'true'){ + $at += 3; + next_chr; + return $JSON::PP::true; + } + elsif($word eq 'null'){ + $at += 3; + next_chr; + return undef; + } + elsif($word eq 'fals'){ + $at += 3; + if(substr($text,$at,1) eq 'e'){ + $at++; + next_chr; + return $JSON::PP::false; + } + } + + $at--; # for decode_error report + + decode_error("'null' expected") if ($word =~ /^n/); + decode_error("'true' expected") if ($word =~ /^t/); + decode_error("'false' expected") if ($word =~ /^f/); + decode_error("malformed JSON string, neither array, object, number, string or atom"); + } + + + sub number { + my $n = ''; + my $v; + my $is_dec; + my $is_exp; + + if($ch eq '-'){ + $n = '-'; + next_chr; + if (!defined $ch or $ch !~ /\d/) { + decode_error("malformed number (no digits after initial minus)"); + } + } + + # According to RFC4627, hex or oct digits are invalid. + if($ch eq '0'){ + my $peek = substr($text,$at,1); + if($peek =~ /^[0-9a-dfA-DF]/){ # e may be valid (exponential) + decode_error("malformed number (leading zero must not be followed by another digit)"); + } + $n .= $ch; + next_chr; + } + + while(defined $ch and $ch =~ /\d/){ + $n .= $ch; + next_chr; + } + + if(defined $ch and $ch eq '.'){ + $n .= '.'; + $is_dec = 1; + + next_chr; + if (!defined $ch or $ch !~ /\d/) { + decode_error("malformed number (no digits after decimal point)"); + } + else { + $n .= $ch; + } + + while(defined(next_chr) and $ch =~ /\d/){ + $n .= $ch; + } + } + + if(defined $ch and ($ch eq 'e' or $ch eq 'E')){ + $n .= $ch; + $is_exp = 1; + next_chr; + + if(defined($ch) and ($ch eq '+' or $ch eq '-')){ + $n .= $ch; + next_chr; + if (!defined $ch or $ch =~ /\D/) { + decode_error("malformed number (no digits after exp sign)"); + } + $n .= $ch; + } + elsif(defined($ch) and $ch =~ /\d/){ + $n .= $ch; + } + else { + decode_error("malformed number (no digits after exp sign)"); + } + + while(defined(next_chr) and $ch =~ /\d/){ + $n .= $ch; + } + + } + + $v .= $n; + + if ($is_dec or $is_exp) { + if ($allow_bignum) { + require Math::BigFloat; + return Math::BigFloat->new($v); + } + } else { + if (length $v > $max_intsize) { + if ($allow_bignum) { # from Adam Sussman + require Math::BigInt; + return Math::BigInt->new($v); + } + else { + return "$v"; + } + } + } + + return $is_dec ? $v/1.0 : 0+$v; + } + + + sub is_valid_utf8 { + + $utf8_len = $_[0] =~ /[\x00-\x7F]/ ? 1 + : $_[0] =~ /[\xC2-\xDF]/ ? 2 + : $_[0] =~ /[\xE0-\xEF]/ ? 3 + : $_[0] =~ /[\xF0-\xF4]/ ? 4 + : 0 + ; + + return unless $utf8_len; + + my $is_valid_utf8 = substr($text, $at - 1, $utf8_len); + + return ( $is_valid_utf8 =~ /^(?: + [\x00-\x7F] + |[\xC2-\xDF][\x80-\xBF] + |[\xE0][\xA0-\xBF][\x80-\xBF] + |[\xE1-\xEC][\x80-\xBF][\x80-\xBF] + |[\xED][\x80-\x9F][\x80-\xBF] + |[\xEE-\xEF][\x80-\xBF][\x80-\xBF] + |[\xF0][\x90-\xBF][\x80-\xBF][\x80-\xBF] + |[\xF1-\xF3][\x80-\xBF][\x80-\xBF][\x80-\xBF] + |[\xF4][\x80-\x8F][\x80-\xBF][\x80-\xBF] + )$/x ) ? $is_valid_utf8 : ''; + } + + + sub decode_error { + my $error = shift; + my $no_rep = shift; + my $str = defined $text ? substr($text, $at) : ''; + my $mess = ''; + my $type = 'U*'; + + if ( OLD_PERL ) { + my $type = $] < 5.006 ? 'C*' + : utf8::is_utf8( $str ) ? 'U*' # 5.6 + : 'C*' + ; + } + + for my $c ( unpack( $type, $str ) ) { # emulate pv_uni_display() ? + $mess .= $c == 0x07 ? '\a' + : $c == 0x09 ? '\t' + : $c == 0x0a ? '\n' + : $c == 0x0d ? '\r' + : $c == 0x0c ? '\f' + : $c < 0x20 ? sprintf('\x{%x}', $c) + : $c == 0x5c ? '\\\\' + : $c < 0x80 ? chr($c) + : sprintf('\x{%x}', $c) + ; + if ( length $mess >= 20 ) { + $mess .= '...'; + last; + } + } + + unless ( length $mess ) { + $mess = '(end of string)'; + } + + Carp::croak ( + $no_rep ? "$error" : "$error, at character offset $at (before \"$mess\")" + ); + + } + + + sub _json_object_hook { + my $o = $_[0]; + my @ks = keys %{$o}; + + if ( $cb_sk_object and @ks == 1 and exists $cb_sk_object->{ $ks[0] } and ref $cb_sk_object->{ $ks[0] } ) { + my @val = $cb_sk_object->{ $ks[0] }->( $o->{$ks[0]} ); + if (@val == 1) { + return $val[0]; + } + } + + my @val = $cb_object->($o) if ($cb_object); + if (@val == 0 or @val > 1) { + return $o; + } + else { + return $val[0]; + } + } + + + sub PP_decode_box { + { + text => $text, + at => $at, + ch => $ch, + len => $len, + depth => $depth, + encoding => $encoding, + is_valid_utf8 => $is_valid_utf8, + }; + } + +} # PARSE + + +sub _decode_surrogates { # from perlunicode + my $uni = 0x10000 + (hex($_[0]) - 0xD800) * 0x400 + (hex($_[1]) - 0xDC00); + my $un = pack('U*', $uni); + utf8::encode( $un ); + return $un; +} + + +sub _decode_unicode { + my $un = pack('U', hex shift); + utf8::encode( $un ); + return $un; +} + +# +# Setup for various Perl versions (the code from JSON::PP58) +# + +BEGIN { + + unless ( defined &utf8::is_utf8 ) { + require Encode; + *utf8::is_utf8 = *Encode::is_utf8; + } + + if ( !OLD_PERL ) { + *JSON::PP::JSON_PP_encode_ascii = \&_encode_ascii; + *JSON::PP::JSON_PP_encode_latin1 = \&_encode_latin1; + *JSON::PP::JSON_PP_decode_surrogates = \&_decode_surrogates; + *JSON::PP::JSON_PP_decode_unicode = \&_decode_unicode; + + if ($] < 5.008003) { # join() in 5.8.0 - 5.8.2 is broken. + package # hide from PAUSE + JSON::PP; + require subs; + subs->import('join'); + eval q| + sub join { + return '' if (@_ < 2); + my $j = shift; + my $str = shift; + for (@_) { $str .= $j . $_; } + return $str; + } + |; + } + } + + + sub JSON::PP::incr_parse { + local $Carp::CarpLevel = 1; + ( $_[0]->{_incr_parser} ||= JSON::PP::IncrParser->new )->incr_parse( @_ ); + } + + + sub JSON::PP::incr_skip { + ( $_[0]->{_incr_parser} ||= JSON::PP::IncrParser->new )->incr_skip; + } + + + sub JSON::PP::incr_reset { + ( $_[0]->{_incr_parser} ||= JSON::PP::IncrParser->new )->incr_reset; + } + + eval q{ + sub JSON::PP::incr_text : lvalue { + $_[0]->{_incr_parser} ||= JSON::PP::IncrParser->new; + + if ( $_[0]->{_incr_parser}->{incr_pos} ) { + Carp::croak("incr_text cannot be called when the incremental parser already started parsing"); + } + $_[0]->{_incr_parser}->{incr_text}; + } + } if ( $] >= 5.006 ); + +} # Setup for various Perl versions (the code from JSON::PP58) + + +############################### +# Utilities +# + +BEGIN { + eval 'require Scalar::Util'; + unless($@){ + *JSON::PP::blessed = \&Scalar::Util::blessed; + *JSON::PP::reftype = \&Scalar::Util::reftype; + *JSON::PP::refaddr = \&Scalar::Util::refaddr; + } + else{ # This code is from Scalar::Util. + # warn $@; + eval 'sub UNIVERSAL::a_sub_not_likely_to_be_here { ref($_[0]) }'; + *JSON::PP::blessed = sub { + local($@, $SIG{__DIE__}, $SIG{__WARN__}); + ref($_[0]) ? eval { $_[0]->a_sub_not_likely_to_be_here } : undef; + }; + require B; + my %tmap = qw( + B::NULL SCALAR + B::HV HASH + B::AV ARRAY + B::CV CODE + B::IO IO + B::GV GLOB + B::REGEXP REGEXP + ); + *JSON::PP::reftype = sub { + my $r = shift; + + return undef unless length(ref($r)); + + my $t = ref(B::svref_2object($r)); + + return + exists $tmap{$t} ? $tmap{$t} + : length(ref($$r)) ? 'REF' + : 'SCALAR'; + }; + *JSON::PP::refaddr = sub { + return undef unless length(ref($_[0])); + + my $addr; + if(defined(my $pkg = blessed($_[0]))) { + $addr .= bless $_[0], 'Scalar::Util::Fake'; + bless $_[0], $pkg; + } + else { + $addr .= $_[0] + } + + $addr =~ /0x(\w+)/; + local $^W; + #no warnings 'portable'; + hex($1); + } + } +} + + +# shamelessly copied and modified from JSON::XS code. + +$JSON::PP::true = do { bless \(my $dummy = 1), "JSON::PP::Boolean" }; +$JSON::PP::false = do { bless \(my $dummy = 0), "JSON::PP::Boolean" }; + +sub is_bool { defined $_[0] and UNIVERSAL::isa($_[0], "JSON::PP::Boolean"); } + +sub true { $JSON::PP::true } +sub false { $JSON::PP::false } +sub null { undef; } + +############################### + +package # hide from PAUSE + JSON::PP::IncrParser; + +use strict; + +use constant INCR_M_WS => 0; # initial whitespace skipping +use constant INCR_M_STR => 1; # inside string +use constant INCR_M_BS => 2; # inside backslash +use constant INCR_M_JSON => 3; # outside anything, count nesting +use constant INCR_M_C0 => 4; +use constant INCR_M_C1 => 5; + +$JSON::backportPP::IncrParser::VERSION = '1.01'; + +sub new { + my ( $class ) = @_; + + bless { + incr_nest => 0, + incr_text => undef, + incr_pos => 0, + incr_mode => 0, + }, $class; +} + + +sub incr_parse { + my ( $self, $coder, $text ) = @_; + + $self->{incr_text} = '' unless ( defined $self->{incr_text} ); + + if ( defined $text ) { + if ( utf8::is_utf8( $text ) and !utf8::is_utf8( $self->{incr_text} ) ) { + utf8::upgrade( $self->{incr_text} ) ; + utf8::decode( $self->{incr_text} ) ; + } + $self->{incr_text} .= $text; + } + + if ( defined wantarray ) { + my $max_size = $coder->get_max_size; + my $p = $self->{incr_pos}; + my @ret; + { + do { + unless ( $self->{incr_nest} <= 0 and $self->{incr_mode} == INCR_M_JSON ) { + $self->_incr_parse( $coder ); + + if ( $max_size and $self->{incr_pos} > $max_size ) { + Carp::croak("attempted decode of JSON text of $self->{incr_pos} bytes size, but max_size is set to $max_size"); + } + unless ( $self->{incr_nest} <= 0 and $self->{incr_mode} == INCR_M_JSON ) { + # as an optimisation, do not accumulate white space in the incr buffer + if ( $self->{incr_mode} == INCR_M_WS and $self->{incr_pos} ) { + $self->{incr_pos} = 0; + $self->{incr_text} = ''; + } + last; + } + } + + my ($obj, $offset) = $coder->PP_decode_json( $self->{incr_text}, 0x00000001 ); + push @ret, $obj; + use bytes; + $self->{incr_text} = substr( $self->{incr_text}, $offset || 0 ); + $self->{incr_pos} = 0; + $self->{incr_nest} = 0; + $self->{incr_mode} = 0; + last unless wantarray; + } while ( wantarray ); + } + + if ( wantarray ) { + return @ret; + } + else { # in scalar context + return $ret[0] ? $ret[0] : undef; + } + } +} + + +sub _incr_parse { + my ($self, $coder) = @_; + my $text = $self->{incr_text}; + my $len = length $text; + my $p = $self->{incr_pos}; + +INCR_PARSE: + while ( $len > $p ) { + my $s = substr( $text, $p, 1 ); + last INCR_PARSE unless defined $s; + my $mode = $self->{incr_mode}; + + if ( $mode == INCR_M_WS ) { + while ( $len > $p ) { + $s = substr( $text, $p, 1 ); + last INCR_PARSE unless defined $s; + if ( ord($s) > 0x20 ) { + if ( $s eq '#' ) { + $self->{incr_mode} = INCR_M_C0; + redo INCR_PARSE; + } else { + $self->{incr_mode} = INCR_M_JSON; + redo INCR_PARSE; + } + } + $p++; + } + } elsif ( $mode == INCR_M_BS ) { + $p++; + $self->{incr_mode} = INCR_M_STR; + redo INCR_PARSE; + } elsif ( $mode == INCR_M_C0 or $mode == INCR_M_C1 ) { + while ( $len > $p ) { + $s = substr( $text, $p, 1 ); + last INCR_PARSE unless defined $s; + if ( $s eq "\n" ) { + $self->{incr_mode} = $self->{incr_mode} == INCR_M_C0 ? INCR_M_WS : INCR_M_JSON; + last; + } + $p++; + } + next; + } elsif ( $mode == INCR_M_STR ) { + while ( $len > $p ) { + $s = substr( $text, $p, 1 ); + last INCR_PARSE unless defined $s; + if ( $s eq '"' ) { + $p++; + $self->{incr_mode} = INCR_M_JSON; + + last INCR_PARSE unless $self->{incr_nest}; + redo INCR_PARSE; + } + elsif ( $s eq '\\' ) { + $p++; + if ( !defined substr($text, $p, 1) ) { + $self->{incr_mode} = INCR_M_BS; + last INCR_PARSE; + } + } + $p++; + } + } elsif ( $mode == INCR_M_JSON ) { + while ( $len > $p ) { + $s = substr( $text, $p++, 1 ); + if ( $s eq "\x00" ) { + $p--; + last INCR_PARSE; + } elsif ( $s eq "\x09" or $s eq "\x0a" or $s eq "\x0d" or $s eq "\x20" ) { + if ( !$self->{incr_nest} ) { + $p--; # do not eat the whitespace, let the next round do it + last INCR_PARSE; + } + next; + } elsif ( $s eq '"' ) { + $self->{incr_mode} = INCR_M_STR; + redo INCR_PARSE; + } elsif ( $s eq '[' or $s eq '{' ) { + if ( ++$self->{incr_nest} > $coder->get_max_depth ) { + Carp::croak('json text or perl structure exceeds maximum nesting level (max_depth set too low?)'); + } + next; + } elsif ( $s eq ']' or $s eq '}' ) { + if ( --$self->{incr_nest} <= 0 ) { + last INCR_PARSE; + } + } elsif ( $s eq '#' ) { + $self->{incr_mode} = INCR_M_C1; + redo INCR_PARSE; + } + } + } + } + + $self->{incr_pos} = $p; + $self->{incr_parsing} = $p ? 1 : 0; # for backward compatibility +} + + +sub incr_text { + if ( $_[0]->{incr_pos} ) { + Carp::croak("incr_text cannot be called when the incremental parser already started parsing"); + } + $_[0]->{incr_text}; +} + + +sub incr_skip { + my $self = shift; + $self->{incr_text} = substr( $self->{incr_text}, $self->{incr_pos} ); + $self->{incr_pos} = 0; + $self->{incr_mode} = 0; + $self->{incr_nest} = 0; +} + + +sub incr_reset { + my $self = shift; + $self->{incr_text} = undef; + $self->{incr_pos} = 0; + $self->{incr_mode} = 0; + $self->{incr_nest} = 0; +} + +############################### + + +1; +__END__ +=pod + +=head1 NAME + +JSON::PP - JSON::XS compatible pure-Perl module. + +=head1 SYNOPSIS + + use JSON::PP; + + # exported functions, they croak on error + # and expect/generate UTF-8 + + $utf8_encoded_json_text = encode_json $perl_hash_or_arrayref; + $perl_hash_or_arrayref = decode_json $utf8_encoded_json_text; + + # OO-interface + + $json = JSON::PP->new->ascii->pretty->allow_nonref; + + $pretty_printed_json_text = $json->encode( $perl_scalar ); + $perl_scalar = $json->decode( $json_text ); + + # Note that JSON version 2.0 and above will automatically use + # JSON::XS or JSON::PP, so you should be able to just: + + use JSON; + + +=head1 VERSION + + 2.91_04 + +=head1 DESCRIPTION + +JSON::PP is a pure perl JSON decoder/encoder (as of RFC4627, which +we know is obsolete but we still stick to; see below for an option +to support part of RFC7159), and (almost) compatible to much +faster L<JSON::XS> written by Marc Lehmann in C. JSON::PP works as +a fallback module when you use L<JSON> module without having +installed JSON::XS. + +Because of this fallback feature of JSON.pm, JSON::PP tries not to +be more JavaScript-friendly than JSON::XS (i.e. not to escape extra +characters such as U+2028 and U+2029 nor support RFC7159/ECMA-404), +in order for you not to lose such JavaScript-friendliness silently +when you use JSON.pm and install JSON::XS for speed or by accident. +If you need JavaScript-friendly RFC7159-compliant pure perl module, +try L<JSON::Tiny>, which is derived from L<Mojolicious> web +framework and is also smaller and faster than JSON::PP. + +JSON::PP has been in the Perl core since Perl 5.14, mainly for +CPAN toolchain modules to parse META.json. + +=head1 FUNCTIONAL INTERFACE + +This section is taken from JSON::XS almost verbatim. C<encode_json> +and C<decode_json> are exported by default. + +=head2 encode_json + + $json_text = encode_json $perl_scalar + +Converts the given Perl data structure to a UTF-8 encoded, binary string +(that is, the string contains octets only). Croaks on error. + +This function call is functionally identical to: + + $json_text = JSON::PP->new->utf8->encode($perl_scalar) + +Except being faster. + +=head2 decode_json + + $perl_scalar = decode_json $json_text + +The opposite of C<encode_json>: expects an UTF-8 (binary) string and tries +to parse that as an UTF-8 encoded JSON text, returning the resulting +reference. Croaks on error. + +This function call is functionally identical to: + + $perl_scalar = JSON::PP->new->utf8->decode($json_text) + +Except being faster. + +=head2 JSON::PP::is_bool + + $is_boolean = JSON::PP::is_bool($scalar) + +Returns true if the passed scalar represents either JSON::PP::true or +JSON::PP::false, two constants that act like C<1> and C<0> respectively +and are also used to represent JSON C<true> and C<false> in Perl strings. + +See L<MAPPING>, below, for more information on how JSON values are mapped to +Perl. + +=head1 OBJECT-ORIENTED INTERFACE + +This section is also taken from JSON::XS. + +The object oriented interface lets you configure your own encoding or +decoding style, within the limits of supported formats. + +=head2 new + + $json = JSON::PP->new + +Creates a new JSON::PP object that can be used to de/encode JSON +strings. All boolean flags described below are by default I<disabled>. + +The mutators for flags all return the JSON::PP object again and thus calls can +be chained: + + my $json = JSON::PP->new->utf8->space_after->encode({a => [1,2]}) + => {"a": [1, 2]} + +=head2 ascii + + $json = $json->ascii([$enable]) + + $enabled = $json->get_ascii + +If C<$enable> is true (or missing), then the C<encode> method will not +generate characters outside the code range C<0..127> (which is ASCII). Any +Unicode characters outside that range will be escaped using either a +single \uXXXX (BMP characters) or a double \uHHHH\uLLLLL escape sequence, +as per RFC4627. The resulting encoded JSON text can be treated as a native +Unicode string, an ascii-encoded, latin1-encoded or UTF-8 encoded string, +or any other superset of ASCII. + +If C<$enable> is false, then the C<encode> method will not escape Unicode +characters unless required by the JSON syntax or other flags. This results +in a faster and more compact format. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this document. + +The main use for this flag is to produce JSON texts that can be +transmitted over a 7-bit channel, as the encoded JSON texts will not +contain any 8 bit characters. + + JSON::PP->new->ascii(1)->encode([chr 0x10401]) + => ["\ud801\udc01"] + +=head2 latin1 + + $json = $json->latin1([$enable]) + + $enabled = $json->get_latin1 + +If C<$enable> is true (or missing), then the C<encode> method will encode +the resulting JSON text as latin1 (or iso-8859-1), escaping any characters +outside the code range C<0..255>. The resulting string can be treated as a +latin1-encoded JSON text or a native Unicode string. The C<decode> method +will not be affected in any way by this flag, as C<decode> by default +expects Unicode, which is a strict superset of latin1. + +If C<$enable> is false, then the C<encode> method will not escape Unicode +characters unless required by the JSON syntax or other flags. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this document. + +The main use for this flag is efficiently encoding binary data as JSON +text, as most octets will not be escaped, resulting in a smaller encoded +size. The disadvantage is that the resulting JSON text is encoded +in latin1 (and must correctly be treated as such when storing and +transferring), a rare encoding for JSON. It is therefore most useful when +you want to store data structures known to contain binary data efficiently +in files or databases, not when talking to other JSON encoders/decoders. + + JSON::PP->new->latin1->encode (["\x{89}\x{abc}"] + => ["\x{89}\\u0abc"] # (perl syntax, U+abc escaped, U+89 not) + +=head2 utf8 + + $json = $json->utf8([$enable]) + + $enabled = $json->get_utf8 + +If C<$enable> is true (or missing), then the C<encode> method will encode +the JSON result into UTF-8, as required by many protocols, while the +C<decode> method expects to be handled an UTF-8-encoded string. Please +note that UTF-8-encoded strings do not contain any characters outside the +range C<0..255>, they are thus useful for bytewise/binary I/O. In future +versions, enabling this option might enable autodetection of the UTF-16 +and UTF-32 encoding families, as described in RFC4627. + +If C<$enable> is false, then the C<encode> method will return the JSON +string as a (non-encoded) Unicode string, while C<decode> expects thus a +Unicode string. Any decoding or encoding (e.g. to UTF-8 or UTF-16) needs +to be done yourself, e.g. using the Encode module. + +See also the section I<ENCODING/CODESET FLAG NOTES> later in this document. + +Example, output UTF-16BE-encoded JSON: + + use Encode; + $jsontext = encode "UTF-16BE", JSON::PP->new->encode ($object); + +Example, decode UTF-32LE-encoded JSON: + + use Encode; + $object = JSON::PP->new->decode (decode "UTF-32LE", $jsontext); + +=head2 pretty + + $json = $json->pretty([$enable]) + +This enables (or disables) all of the C<indent>, C<space_before> and +C<space_after> (and in the future possibly more) flags in one call to +generate the most readable (or most compact) form possible. + +=head2 indent + + $json = $json->indent([$enable]) + + $enabled = $json->get_indent + +If C<$enable> is true (or missing), then the C<encode> method will use a multiline +format as output, putting every array member or object/hash key-value pair +into its own line, indenting them properly. + +If C<$enable> is false, no newlines or indenting will be produced, and the +resulting JSON text is guaranteed not to contain any C<newlines>. + +This setting has no effect when decoding JSON texts. + +The default indent space length is three. +You can use C<indent_length> to change the length. + +=head2 space_before + + $json = $json->space_before([$enable]) + + $enabled = $json->get_space_before + +If C<$enable> is true (or missing), then the C<encode> method will add an extra +optional space before the C<:> separating keys from values in JSON objects. + +If C<$enable> is false, then the C<encode> method will not add any extra +space at those places. + +This setting has no effect when decoding JSON texts. You will also +most likely combine this setting with C<space_after>. + +Example, space_before enabled, space_after and indent disabled: + + {"key" :"value"} + +=head2 space_after + + $json = $json->space_after([$enable]) + + $enabled = $json->get_space_after + +If C<$enable> is true (or missing), then the C<encode> method will add an extra +optional space after the C<:> separating keys from values in JSON objects +and extra whitespace after the C<,> separating key-value pairs and array +members. + +If C<$enable> is false, then the C<encode> method will not add any extra +space at those places. + +This setting has no effect when decoding JSON texts. + +Example, space_before and indent disabled, space_after enabled: + + {"key": "value"} + +=head2 relaxed + + $json = $json->relaxed([$enable]) + + $enabled = $json->get_relaxed + +If C<$enable> is true (or missing), then C<decode> will accept some +extensions to normal JSON syntax (see below). C<encode> will not be +affected in anyway. I<Be aware that this option makes you accept invalid +JSON texts as if they were valid!>. I suggest only to use this option to +parse application-specific files written by humans (configuration files, +resource files etc.) + +If C<$enable> is false (the default), then C<decode> will only accept +valid JSON texts. + +Currently accepted extensions are: + +=over 4 + +=item * list items can have an end-comma + +JSON I<separates> array elements and key-value pairs with commas. This +can be annoying if you write JSON texts manually and want to be able to +quickly append elements, so this extension accepts comma at the end of +such items not just between them: + + [ + 1, + 2, <- this comma not normally allowed + ] + { + "k1": "v1", + "k2": "v2", <- this comma not normally allowed + } + +=item * shell-style '#'-comments + +Whenever JSON allows whitespace, shell-style comments are additionally +allowed. They are terminated by the first carriage-return or line-feed +character, after which more white-space and comments are allowed. + + [ + 1, # this comment not allowed in JSON + # neither this one... + ] + +=item * C-style multiple-line '/* */'-comments (JSON::PP only) + +Whenever JSON allows whitespace, C-style multiple-line comments are additionally +allowed. Everything between C</*> and C<*/> is a comment, after which +more white-space and comments are allowed. + + [ + 1, /* this comment not allowed in JSON */ + /* neither this one... */ + ] + +=item * C++-style one-line '//'-comments (JSON::PP only) + +Whenever JSON allows whitespace, C++-style one-line comments are additionally +allowed. They are terminated by the first carriage-return or line-feed +character, after which more white-space and comments are allowed. + + [ + 1, // this comment not allowed in JSON + // neither this one... + ] + +=back + +=head2 canonical + + $json = $json->canonical([$enable]) + + $enabled = $json->get_canonical + +If C<$enable> is true (or missing), then the C<encode> method will output JSON objects +by sorting their keys. This is adding a comparatively high overhead. + +If C<$enable> is false, then the C<encode> method will output key-value +pairs in the order Perl stores them (which will likely change between runs +of the same script, and can change even within the same run from 5.18 +onwards). + +This option is useful if you want the same data structure to be encoded as +the same JSON text (given the same overall settings). If it is disabled, +the same hash might be encoded differently even if contains the same data, +as key-value pairs have no inherent ordering in Perl. + +This setting has no effect when decoding JSON texts. + +This setting has currently no effect on tied hashes. + +=head2 allow_nonref + + $json = $json->allow_nonref([$enable]) + + $enabled = $json->get_allow_nonref + +If C<$enable> is true (or missing), then the C<encode> method can convert a +non-reference into its corresponding string, number or null JSON value, +which is an extension to RFC4627. Likewise, C<decode> will accept those JSON +values instead of croaking. + +If C<$enable> is false, then the C<encode> method will croak if it isn't +passed an arrayref or hashref, as JSON texts must either be an object +or array. Likewise, C<decode> will croak if given something that is not a +JSON object or array. + +Example, encode a Perl scalar as JSON value with enabled C<allow_nonref>, +resulting in an invalid JSON text: + + JSON::PP->new->allow_nonref->encode ("Hello, World!") + => "Hello, World!" + +=head2 allow_unknown + + $json = $json->allow_unknown ([$enable]) + + $enabled = $json->get_allow_unknown + +If C<$enable> is true (or missing), then C<encode> will I<not> throw an +exception when it encounters values it cannot represent in JSON (for +example, filehandles) but instead will encode a JSON C<null> value. Note +that blessed objects are not included here and are handled separately by +c<allow_blessed>. + +If C<$enable> is false (the default), then C<encode> will throw an +exception when it encounters anything it cannot encode as JSON. + +This option does not affect C<decode> in any way, and it is recommended to +leave it off unless you know your communications partner. + +=head2 allow_blessed + + $json = $json->allow_blessed([$enable]) + + $enabled = $json->get_allow_blessed + +See L<OBJECT SERIALISATION> for details. + +If C<$enable> is true (or missing), then the C<encode> method will not +barf when it encounters a blessed reference that it cannot convert +otherwise. Instead, a JSON C<null> value is encoded instead of the object. + +If C<$enable> is false (the default), then C<encode> will throw an +exception when it encounters a blessed object that it cannot convert +otherwise. + +This setting has no effect on C<decode>. + +=head2 convert_blessed + + $json = $json->convert_blessed([$enable]) + + $enabled = $json->get_convert_blessed + +See L<OBJECT SERIALISATION> for details. + +If C<$enable> is true (or missing), then C<encode>, upon encountering a +blessed object, will check for the availability of the C<TO_JSON> method +on the object's class. If found, it will be called in scalar context and +the resulting scalar will be encoded instead of the object. + +The C<TO_JSON> method may safely call die if it wants. If C<TO_JSON> +returns other blessed objects, those will be handled in the same +way. C<TO_JSON> must take care of not causing an endless recursion cycle +(== crash) in this case. The name of C<TO_JSON> was chosen because other +methods called by the Perl core (== not by the user of the object) are +usually in upper case letters and to avoid collisions with any C<to_json> +function or method. + +If C<$enable> is false (the default), then C<encode> will not consider +this type of conversion. + +This setting has no effect on C<decode>. + +=head2 filter_json_object + + $json = $json->filter_json_object([$coderef]) + +When C<$coderef> is specified, it will be called from C<decode> each +time it decodes a JSON object. The only argument is a reference to the +newly-created hash. If the code references returns a single scalar (which +need not be a reference), this value (i.e. a copy of that scalar to avoid +aliasing) is inserted into the deserialised data structure. If it returns +an empty list (NOTE: I<not> C<undef>, which is a valid scalar), the +original deserialised hash will be inserted. This setting can slow down +decoding considerably. + +When C<$coderef> is omitted or undefined, any existing callback will +be removed and C<decode> will not change the deserialised hash in any +way. + +Example, convert all JSON objects into the integer 5: + + my $js = JSON::PP->new->filter_json_object (sub { 5 }); + # returns [5] + $js->decode ('[{}]'); # the given subroutine takes a hash reference. + # throw an exception because allow_nonref is not enabled + # so a lone 5 is not allowed. + $js->decode ('{"a":1, "b":2}'); + +=head2 filter_json_single_key_object + + $json = $json->filter_json_single_key_object($key [=> $coderef]) + +Works remotely similar to C<filter_json_object>, but is only called for +JSON objects having a single key named C<$key>. + +This C<$coderef> is called before the one specified via +C<filter_json_object>, if any. It gets passed the single value in the JSON +object. If it returns a single value, it will be inserted into the data +structure. If it returns nothing (not even C<undef> but the empty list), +the callback from C<filter_json_object> will be called next, as if no +single-key callback were specified. + +If C<$coderef> is omitted or undefined, the corresponding callback will be +disabled. There can only ever be one callback for a given key. + +As this callback gets called less often then the C<filter_json_object> +one, decoding speed will not usually suffer as much. Therefore, single-key +objects make excellent targets to serialise Perl objects into, especially +as single-key JSON objects are as close to the type-tagged value concept +as JSON gets (it's basically an ID/VALUE tuple). Of course, JSON does not +support this in any way, so you need to make sure your data never looks +like a serialised Perl hash. + +Typical names for the single object key are C<__class_whatever__>, or +C<$__dollars_are_rarely_used__$> or C<}ugly_brace_placement>, or even +things like C<__class_md5sum(classname)__>, to reduce the risk of clashing +with real hashes. + +Example, decode JSON objects of the form C<< { "__widget__" => <id> } >> +into the corresponding C<< $WIDGET{<id>} >> object: + + # return whatever is in $WIDGET{5}: + JSON::PP + ->new + ->filter_json_single_key_object (__widget__ => sub { + $WIDGET{ $_[0] } + }) + ->decode ('{"__widget__": 5') + + # this can be used with a TO_JSON method in some "widget" class + # for serialisation to json: + sub WidgetBase::TO_JSON { + my ($self) = @_; + + unless ($self->{id}) { + $self->{id} = ..get..some..id..; + $WIDGET{$self->{id}} = $self; + } + + { __widget__ => $self->{id} } + } + +=head2 shrink + + $json = $json->shrink([$enable]) + + $enabled = $json->get_shrink + +If C<$enable> is true (or missing), the string returned by C<encode> will +be shrunk (i.e. downgraded if possible). + +The actual definition of what shrink does might change in future versions, +but it will always try to save space at the expense of time. + +If C<$enable> is false, then JSON::PP does nothing. + +=head2 max_depth + + $json = $json->max_depth([$maximum_nesting_depth]) + + $max_depth = $json->get_max_depth + +Sets the maximum nesting level (default C<512>) accepted while encoding +or decoding. If a higher nesting level is detected in JSON text or a Perl +data structure, then the encoder and decoder will stop and croak at that +point. + +Nesting level is defined by number of hash- or arrayrefs that the encoder +needs to traverse to reach a given point or the number of C<{> or C<[> +characters without their matching closing parenthesis crossed to reach a +given character in a string. + +Setting the maximum depth to one disallows any nesting, so that ensures +that the object is only a single hash/object or array. + +If no argument is given, the highest possible setting will be used, which +is rarely useful. + +See L<JSON::XS/SECURITY CONSIDERATIONS> for more info on why this is useful. + +=head2 max_size + + $json = $json->max_size([$maximum_string_size]) + + $max_size = $json->get_max_size + +Set the maximum length a JSON text may have (in bytes) where decoding is +being attempted. The default is C<0>, meaning no limit. When C<decode> +is called on a string that is longer then this many bytes, it will not +attempt to decode the string but throw an exception. This setting has no +effect on C<encode> (yet). + +If no argument is given, the limit check will be deactivated (same as when +C<0> is specified). + +See L<JSON::XS/SECURITY CONSIDERATIONS> for more info on why this is useful. + +=head2 encode + + $json_text = $json->encode($perl_scalar) + +Converts the given Perl value or data structure to its JSON +representation. Croaks on error. + +=head2 decode + + $perl_scalar = $json->decode($json_text) + +The opposite of C<encode>: expects a JSON text and tries to parse it, +returning the resulting simple scalar or reference. Croaks on error. + +=head2 decode_prefix + + ($perl_scalar, $characters) = $json->decode_prefix($json_text) + +This works like the C<decode> method, but instead of raising an exception +when there is trailing garbage after the first JSON object, it will +silently stop parsing there and return the number of characters consumed +so far. + +This is useful if your JSON texts are not delimited by an outer protocol +and you need to know where the JSON text ends. + + JSON::PP->new->decode_prefix ("[1] the tail") + => ([1], 3) + +=head1 FLAGS FOR JSON::PP ONLY + +The following flags and properties are for JSON::PP only. If you use +any of these, you can't make your application run faster by replacing +JSON::PP with JSON::XS. If you need these and also speed boost, +try L<Cpanel::JSON::XS>, a fork of JSON::XS by Reini Urban, which +supports some of these. + +=head2 allow_singlequote + + $json = $json->allow_singlequote([$enable]) + $enabled = $json->get_allow_singlequote + +If C<$enable> is true (or missing), then C<decode> will accept +invalid JSON texts that contain strings that begin and end with +single quotation marks. C<encode> will not be affected in anyway. +I<Be aware that this option makes you accept invalid JSON texts +as if they were valid!>. I suggest only to use this option to +parse application-specific files written by humans (configuration +files, resource files etc.) + +If C<$enable> is false (the default), then C<decode> will only accept +valid JSON texts. + + $json->allow_singlequote->decode(qq|{"foo":'bar'}|); + $json->allow_singlequote->decode(qq|{'foo':"bar"}|); + $json->allow_singlequote->decode(qq|{'foo':'bar'}|); + +=head2 allow_barekey + + $json = $json->allow_barekey([$enable]) + $enabled = $json->get_allow_barekey + +If C<$enable> is true (or missing), then C<decode> will accept +invalid JSON texts that contain JSON objects whose names don't +begin and end with quotation marks. C<encode> will not be affected +in anyway. I<Be aware that this option makes you accept invalid JSON +texts as if they were valid!>. I suggest only to use this option to +parse application-specific files written by humans (configuration +files, resource files etc.) + +If C<$enable> is false (the default), then C<decode> will only accept +valid JSON texts. + + $json->allow_barekey->decode(qq|{foo:"bar"}|); + +=head2 allow_bignum + + $json = $json->allow_bignum([$enable]) + $enabled = $json->get_allow_bignum + +If C<$enable> is true (or missing), then C<decode> will convert +big integers Perl cannot handle as integer into L<Math::BigInt> +objects and convert floating numbers into L<Math::BigFloat> +objects. C<encode> will convert C<Math::BigInt> and C<Math::BigFloat> +objects into JSON numbers. + + $json->allow_nonref->allow_bignum; + $bigfloat = $json->decode('2.000000000000000000000000001'); + print $json->encode($bigfloat); + # => 2.000000000000000000000000001 + +See also L<MAPPING>. + +=head2 loose + + $json = $json->loose([$enable]) + $enabled = $json->get_loose + +If C<$enable> is true (or missing), then C<decode> will accept +invalid JSON texts that contain unescaped [\x00-\x1f\x22\x5c] +characters. C<encode> will not be affected in anyway. +I<Be aware that this option makes you accept invalid JSON texts +as if they were valid!>. I suggest only to use this option to +parse application-specific files written by humans (configuration +files, resource files etc.) + +If C<$enable> is false (the default), then C<decode> will only accept +valid JSON texts. + + $json->loose->decode(qq|["abc + def"]|); + +=head2 escape_slash + + $json = $json->escape_slash([$enable]) + $enabled = $json->get_escape_slash + +If C<$enable> is true (or missing), then C<encode> will explicitly +escape I<slash> (solidus; C<U+002F>) characters to reduce the risk of +XSS (cross site scripting) that may be caused by C<< </script> >> +in a JSON text, with the cost of bloating the size of JSON texts. + +This option may be useful when you embed JSON in HTML, but embedding +arbitrary JSON in HTML (by some HTML template toolkit or by string +interpolation) is risky in general. You must escape necessary +characters in correct order, depending on the context. + +C<decode> will not be affected in anyway. + +=head2 indent_length + + $json = $json->indent_length($number_of_spaces) + $length = $json->get_indent_length + +This option is only useful when you also enable C<indent> or C<pretty>. + +JSON::XS indents with three spaces when you C<encode> (if requested +by C<indent> or C<pretty>), and the number cannot be changed. +JSON::PP allows you to change/get the number of indent spaces with these +mutator/accessor. The default number of spaces is three (the same as +JSON::XS), and the acceptable range is from C<0> (no indentation; +it'd be better to disable indentation by C<indent(0)>) to C<15>. + +=head2 sort_by + + $json = $json->sort_by($code_ref) + $json = $json->sort_by($subroutine_name) + +If you just want to sort keys (names) in JSON objects when you +C<encode>, enable C<canonical> option (see above) that allows you to +sort object keys alphabetically. + +If you do need to sort non-alphabetically for whatever reasons, +you can give a code reference (or a subroutine name) to C<sort_by>, +then the argument will be passed to Perl's C<sort> built-in function. + +As the sorting is done in the JSON::PP scope, you usually need to +prepend C<JSON::PP::> to the subroutine name, and the special variables +C<$a> and C<$b> used in the subrontine used by C<sort> function. + +Example: + + my %ORDER = (id => 1, class => 2, name => 3); + $json->sort_by(sub { + ($ORDER{$JSON::PP::a} // 999) <=> ($ORDER{$JSON::PP::b} // 999) + or $JSON::PP::a cmp $JSON::PP::b + }); + print $json->encode([ + {name => 'CPAN', id => 1, href => 'http://cpan.org'} + ]); + # [{"id":1,"name":"CPAN","href":"http://cpan.org"}] + +Note that C<sort_by> affects all the plain hashes in the data structure. +If you need finer control, C<tie> necessary hashes with a module that +implements ordered hash (such as L<Hash::Ordered> and L<Tie::IxHash>). +C<canonical> and C<sort_by> don't affect the key order in C<tie>d +hashes. + + use Hash::Ordered; + tie my %hash, 'Hash::Ordered', + (name => 'CPAN', id => 1, href => 'http://cpan.org'); + print $json->encode([\%hash]); + # [{"name":"CPAN","id":1,"href":"http://cpan.org"}] # order is kept + +=head1 INCREMENTAL PARSING + +This section is also taken from JSON::XS. + +In some cases, there is the need for incremental parsing of JSON +texts. While this module always has to keep both JSON text and resulting +Perl data structure in memory at one time, it does allow you to parse a +JSON stream incrementally. It does so by accumulating text until it has +a full JSON object, which it then can decode. This process is similar to +using C<decode_prefix> to see if a full JSON object is available, but +is much more efficient (and can be implemented with a minimum of method +calls). + +JSON::PP will only attempt to parse the JSON text once it is sure it +has enough text to get a decisive result, using a very simple but +truly incremental parser. This means that it sometimes won't stop as +early as the full parser, for example, it doesn't detect mismatched +parentheses. The only thing it guarantees is that it starts decoding as +soon as a syntactically valid JSON text has been seen. This means you need +to set resource limits (e.g. C<max_size>) to ensure the parser will stop +parsing in the presence if syntax errors. + +The following methods implement this incremental parser. + +=head2 incr_parse + + $json->incr_parse( [$string] ) # void context + + $obj_or_undef = $json->incr_parse( [$string] ) # scalar context + + @obj_or_empty = $json->incr_parse( [$string] ) # list context + +This is the central parsing function. It can both append new text and +extract objects from the stream accumulated so far (both of these +functions are optional). + +If C<$string> is given, then this string is appended to the already +existing JSON fragment stored in the C<$json> object. + +After that, if the function is called in void context, it will simply +return without doing anything further. This can be used to add more text +in as many chunks as you want. + +If the method is called in scalar context, then it will try to extract +exactly I<one> JSON object. If that is successful, it will return this +object, otherwise it will return C<undef>. If there is a parse error, +this method will croak just as C<decode> would do (one can then use +C<incr_skip> to skip the erroneous part). This is the most common way of +using the method. + +And finally, in list context, it will try to extract as many objects +from the stream as it can find and return them, or the empty list +otherwise. For this to work, there must be no separators (other than +whitespace) between the JSON objects or arrays, instead they must be +concatenated back-to-back. If an error occurs, an exception will be +raised as in the scalar context case. Note that in this case, any +previously-parsed JSON texts will be lost. + +Example: Parse some JSON arrays/objects in a given string and return +them. + + my @objs = JSON::PP->new->incr_parse ("[5][7][1,2]"); + +=head2 incr_text + + $lvalue_string = $json->incr_text + +This method returns the currently stored JSON fragment as an lvalue, that +is, you can manipulate it. This I<only> works when a preceding call to +C<incr_parse> in I<scalar context> successfully returned an object. Under +all other circumstances you must not call this function (I mean it. +although in simple tests it might actually work, it I<will> fail under +real world conditions). As a special exception, you can also call this +method before having parsed anything. + +That means you can only use this function to look at or manipulate text +before or after complete JSON objects, not while the parser is in the +middle of parsing a JSON object. + +This function is useful in two cases: a) finding the trailing text after a +JSON object or b) parsing multiple JSON objects separated by non-JSON text +(such as commas). + +=head2 incr_skip + + $json->incr_skip + +This will reset the state of the incremental parser and will remove +the parsed text from the input buffer so far. This is useful after +C<incr_parse> died, in which case the input buffer and incremental parser +state is left unchanged, to skip the text parsed so far and to reset the +parse state. + +The difference to C<incr_reset> is that only text until the parse error +occurred is removed. + +=head2 incr_reset + + $json->incr_reset + +This completely resets the incremental parser, that is, after this call, +it will be as if the parser had never parsed anything. + +This is useful if you want to repeatedly parse JSON objects and want to +ignore any trailing data, which means you have to reset the parser after +each successful decode. + +=head1 MAPPING + +Most of this section is also taken from JSON::XS. + +This section describes how JSON::PP maps Perl values to JSON values and +vice versa. These mappings are designed to "do the right thing" in most +circumstances automatically, preserving round-tripping characteristics +(what you put in comes out as something equivalent). + +For the more enlightened: note that in the following descriptions, +lowercase I<perl> refers to the Perl interpreter, while uppercase I<Perl> +refers to the abstract Perl language itself. + +=head2 JSON -> PERL + +=over 4 + +=item object + +A JSON object becomes a reference to a hash in Perl. No ordering of object +keys is preserved (JSON does not preserve object key ordering itself). + +=item array + +A JSON array becomes a reference to an array in Perl. + +=item string + +A JSON string becomes a string scalar in Perl - Unicode codepoints in JSON +are represented by the same codepoints in the Perl string, so no manual +decoding is necessary. + +=item number + +A JSON number becomes either an integer, numeric (floating point) or +string scalar in perl, depending on its range and any fractional parts. On +the Perl level, there is no difference between those as Perl handles all +the conversion details, but an integer may take slightly less memory and +might represent more values exactly than floating point numbers. + +If the number consists of digits only, JSON::PP will try to represent +it as an integer value. If that fails, it will try to represent it as +a numeric (floating point) value if that is possible without loss of +precision. Otherwise it will preserve the number as a string value (in +which case you lose roundtripping ability, as the JSON number will be +re-encoded to a JSON string). + +Numbers containing a fractional or exponential part will always be +represented as numeric (floating point) values, possibly at a loss of +precision (in which case you might lose perfect roundtripping ability, but +the JSON number will still be re-encoded as a JSON number). + +Note that precision is not accuracy - binary floating point values cannot +represent most decimal fractions exactly, and when converting from and to +floating point, JSON::PP only guarantees precision up to but not including +the least significant bit. + +When C<allow_bignum> is enabled, big integer values and any numeric +values will be converted into L<Math::BigInt> and L<Math::BigFloat> +objects respectively, without becoming string scalars or losing +precision. + +=item true, false + +These JSON atoms become C<JSON::PP::true> and C<JSON::PP::false>, +respectively. They are overloaded to act almost exactly like the numbers +C<1> and C<0>. You can check whether a scalar is a JSON boolean by using +the C<JSON::PP::is_bool> function. + +=item null + +A JSON null atom becomes C<undef> in Perl. + +=item shell-style comments (C<< # I<text> >>) + +As a nonstandard extension to the JSON syntax that is enabled by the +C<relaxed> setting, shell-style comments are allowed. They can start +anywhere outside strings and go till the end of the line. + +=back + + +=head2 PERL -> JSON + +The mapping from Perl to JSON is slightly more difficult, as Perl is a +truly typeless language, so we can only guess which JSON type is meant by +a Perl value. + +=over 4 + +=item hash references + +Perl hash references become JSON objects. As there is no inherent +ordering in hash keys (or JSON objects), they will usually be encoded +in a pseudo-random order. JSON::PP can optionally sort the hash keys +(determined by the I<canonical> flag and/or I<sort_by> property), so +the same data structure will serialise to the same JSON text (given +same settings and version of JSON::PP), but this incurs a runtime +overhead and is only rarely useful, e.g. when you want to compare some +JSON text against another for equality. + +=item array references + +Perl array references become JSON arrays. + +=item other references + +Other unblessed references are generally not allowed and will cause an +exception to be thrown, except for references to the integers C<0> and +C<1>, which get turned into C<false> and C<true> atoms in JSON. You can +also use C<JSON::PP::false> and C<JSON::PP::true> to improve +readability. + + to_json [\0, JSON::PP::true] # yields [false,true] + +=item JSON::PP::true, JSON::PP::false + +These special values become JSON true and JSON false values, +respectively. You can also use C<\1> and C<\0> directly if you want. + +=item JSON::PP::null + +This special value becomes JSON null. + +=item blessed objects + +Blessed objects are not directly representable in JSON, but C<JSON::PP> +allows various ways of handling objects. See L<OBJECT SERIALISATION>, +below, for details. + +=item simple scalars + +Simple Perl scalars (any scalar that is not a reference) are the most +difficult objects to encode: JSON::PP will encode undefined scalars as +JSON C<null> values, scalars that have last been used in a string context +before encoding as JSON strings, and anything else as number value: + + # dump as number + encode_json [2] # yields [2] + encode_json [-3.0e17] # yields [-3e+17] + my $value = 5; encode_json [$value] # yields [5] + + # used as string, so dump as string + print $value; + encode_json [$value] # yields ["5"] + + # undef becomes null + encode_json [undef] # yields [null] + +You can force the type to be a string by stringifying it: + + my $x = 3.1; # some variable containing a number + "$x"; # stringified + $x .= ""; # another, more awkward way to stringify + print $x; # perl does it for you, too, quite often + # (but for older perls) + +You can force the type to be a number by numifying it: + + my $x = "3"; # some variable containing a string + $x += 0; # numify it, ensuring it will be dumped as a number + $x *= 1; # same thing, the choice is yours. + +You cannot currently force the type in other, less obscure, ways. + +Note that numerical precision has the same meaning as under Perl (so +binary to decimal conversion follows the same rules as in Perl, which +can differ to other languages). Also, your perl interpreter might expose +extensions to the floating point numbers of your platform, such as +infinities or NaN's - these cannot be represented in JSON, and it is an +error to pass those in. + +JSON::PP (and JSON::XS) trusts what you pass to C<encode> method +(or C<encode_json> function) is a clean, validated data structure with +values that can be represented as valid JSON values only, because it's +not from an external data source (as opposed to JSON texts you pass to +C<decode> or C<decode_json>, which JSON::PP considers tainted and +doesn't trust). As JSON::PP doesn't know exactly what you and consumers +of your JSON texts want the unexpected values to be (you may want to +convert them into null, or to stringify them with or without +normalisation (string representation of infinities/NaN may vary +depending on platforms), or to croak without conversion), you're advised +to do what you and your consumers need before you encode, and also not +to numify values that may start with values that look like a number +(including infinities/NaN), without validating. + +=back + +=head2 OBJECT SERIALISATION + +As for Perl objects, JSON::PP only supports a pure JSON representation (without the ability to deserialise the object automatically again). + +=head3 SERIALISATION + +What happens when C<JSON::PP> encounters a Perl object depends on the +C<allow_blessed>, C<convert_blessed> and C<allow_bignum> settings, which are +used in this order: + +=over 4 + +=item 1. C<convert_blessed> is enabled and the object has a C<TO_JSON> method. + +In this case, the C<TO_JSON> method of the object is invoked in scalar +context. It must return a single scalar that can be directly encoded into +JSON. This scalar replaces the object in the JSON text. + +For example, the following C<TO_JSON> method will convert all L<URI> +objects to JSON strings when serialised. The fact that these values +originally were L<URI> objects is lost. + + sub URI::TO_JSON { + my ($uri) = @_; + $uri->as_string + } + +=item 2. C<allow_bignum> is enabled and the object is a C<Math::BigInt> or C<Math::BigFloat>. + +The object will be serialised as a JSON number value. + +=item 3. C<allow_blessed> is enabled. + +The object will be serialised as a JSON null value. + +=item 4. none of the above + +If none of the settings are enabled or the respective methods are missing, +C<JSON::PP> throws an exception. + +=back + +=head1 ENCODING/CODESET FLAG NOTES + +This section is taken from JSON::XS. + +The interested reader might have seen a number of flags that signify +encodings or codesets - C<utf8>, C<latin1> and C<ascii>. There seems to be +some confusion on what these do, so here is a short comparison: + +C<utf8> controls whether the JSON text created by C<encode> (and expected +by C<decode>) is UTF-8 encoded or not, while C<latin1> and C<ascii> only +control whether C<encode> escapes character values outside their respective +codeset range. Neither of these flags conflict with each other, although +some combinations make less sense than others. + +Care has been taken to make all flags symmetrical with respect to +C<encode> and C<decode>, that is, texts encoded with any combination of +these flag values will be correctly decoded when the same flags are used +- in general, if you use different flag settings while encoding vs. when +decoding you likely have a bug somewhere. + +Below comes a verbose discussion of these flags. Note that a "codeset" is +simply an abstract set of character-codepoint pairs, while an encoding +takes those codepoint numbers and I<encodes> them, in our case into +octets. Unicode is (among other things) a codeset, UTF-8 is an encoding, +and ISO-8859-1 (= latin 1) and ASCII are both codesets I<and> encodings at +the same time, which can be confusing. + +=over 4 + +=item C<utf8> flag disabled + +When C<utf8> is disabled (the default), then C<encode>/C<decode> generate +and expect Unicode strings, that is, characters with high ordinal Unicode +values (> 255) will be encoded as such characters, and likewise such +characters are decoded as-is, no changes to them will be done, except +"(re-)interpreting" them as Unicode codepoints or Unicode characters, +respectively (to Perl, these are the same thing in strings unless you do +funny/weird/dumb stuff). + +This is useful when you want to do the encoding yourself (e.g. when you +want to have UTF-16 encoded JSON texts) or when some other layer does +the encoding for you (for example, when printing to a terminal using a +filehandle that transparently encodes to UTF-8 you certainly do NOT want +to UTF-8 encode your data first and have Perl encode it another time). + +=item C<utf8> flag enabled + +If the C<utf8>-flag is enabled, C<encode>/C<decode> will encode all +characters using the corresponding UTF-8 multi-byte sequence, and will +expect your input strings to be encoded as UTF-8, that is, no "character" +of the input string must have any value > 255, as UTF-8 does not allow +that. + +The C<utf8> flag therefore switches between two modes: disabled means you +will get a Unicode string in Perl, enabled means you get an UTF-8 encoded +octet/binary string in Perl. + +=item C<latin1> or C<ascii> flags enabled + +With C<latin1> (or C<ascii>) enabled, C<encode> will escape characters +with ordinal values > 255 (> 127 with C<ascii>) and encode the remaining +characters as specified by the C<utf8> flag. + +If C<utf8> is disabled, then the result is also correctly encoded in those +character sets (as both are proper subsets of Unicode, meaning that a +Unicode string with all character values < 256 is the same thing as a +ISO-8859-1 string, and a Unicode string with all character values < 128 is +the same thing as an ASCII string in Perl). + +If C<utf8> is enabled, you still get a correct UTF-8-encoded string, +regardless of these flags, just some more characters will be escaped using +C<\uXXXX> then before. + +Note that ISO-8859-1-I<encoded> strings are not compatible with UTF-8 +encoding, while ASCII-encoded strings are. That is because the ISO-8859-1 +encoding is NOT a subset of UTF-8 (despite the ISO-8859-1 I<codeset> being +a subset of Unicode), while ASCII is. + +Surprisingly, C<decode> will ignore these flags and so treat all input +values as governed by the C<utf8> flag. If it is disabled, this allows you +to decode ISO-8859-1- and ASCII-encoded strings, as both strict subsets of +Unicode. If it is enabled, you can correctly decode UTF-8 encoded strings. + +So neither C<latin1> nor C<ascii> are incompatible with the C<utf8> flag - +they only govern when the JSON output engine escapes a character or not. + +The main use for C<latin1> is to relatively efficiently store binary data +as JSON, at the expense of breaking compatibility with most JSON decoders. + +The main use for C<ascii> is to force the output to not contain characters +with values > 127, which means you can interpret the resulting string +as UTF-8, ISO-8859-1, ASCII, KOI8-R or most about any character set and +8-bit-encoding, and still get the same data structure back. This is useful +when your channel for JSON transfer is not 8-bit clean or the encoding +might be mangled in between (e.g. in mail), and works because ASCII is a +proper subset of most 8-bit and multibyte encodings in use in the world. + +=back + +=head1 SEE ALSO + +The F<json_pp> command line utility for quick experiments. + +L<JSON::XS>, L<Cpanel::JSON::XS>, and L<JSON::Tiny> for faster alternatives. +L<JSON> and L<JSON::MaybeXS> for easy migration. + +L<JSON::backportPP::Compat5005> and L<JSON::backportPP::Compat5006> for older perl users. + +RFC4627 (L<http://www.ietf.org/rfc/rfc4627.txt>) + +=head1 AUTHOR + +Makamaka Hannyaharamitu, E<lt>makamaka[at]cpan.orgE<gt> + + +=head1 COPYRIGHT AND LICENSE + +Copyright 2007-2016 by Makamaka Hannyaharamitu + +This library is free software; you can redistribute it and/or modify +it under the same terms as Perl itself. + +=cut diff --git a/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Boolean.pm b/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Boolean.pm new file mode 100644 index 00000000000..f1c9d526790 --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Boolean.pm @@ -0,0 +1,36 @@ +package # This is JSON::backportPP + JSON::PP::Boolean; + +use strict; +use overload ( + "0+" => sub { ${$_[0]} }, + "++" => sub { $_[0] = ${$_[0]} + 1 }, + "--" => sub { $_[0] = ${$_[0]} - 1 }, + fallback => 1, +); + +$JSON::backportPP::Boolean::VERSION = '2.94'; + +1; + +__END__ + +=head1 NAME + +JSON::PP::Boolean - dummy module providing JSON::PP::Boolean + +=head1 SYNOPSIS + + # do not "use" yourself + +=head1 DESCRIPTION + +This module exists only to provide overload resolution for Storable and similar modules. See +L<JSON::PP> for more info about this class. + +=head1 AUTHOR + +This idea is from L<JSON::XS::Boolean> written by Marc Lehmann <schmorp[at]schmorp.de> + +=cut + diff --git a/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Compat5005.pm b/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Compat5005.pm new file mode 100644 index 00000000000..139990edff0 --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Compat5005.pm @@ -0,0 +1,131 @@ +package # This is JSON::backportPP + JSON::backportPP5005; + +use 5.005; +use strict; + +my @properties; + +$JSON::PP5005::VERSION = '1.10'; + +BEGIN { + + sub utf8::is_utf8 { + 0; # It is considered that UTF8 flag off for Perl 5.005. + } + + sub utf8::upgrade { + } + + sub utf8::downgrade { + 1; # must always return true. + } + + sub utf8::encode { + } + + sub utf8::decode { + } + + *JSON::PP::JSON_PP_encode_ascii = \&_encode_ascii; + *JSON::PP::JSON_PP_encode_latin1 = \&_encode_latin1; + *JSON::PP::JSON_PP_decode_surrogates = \&_decode_surrogates; + *JSON::PP::JSON_PP_decode_unicode = \&_decode_unicode; + + # missing in B module. + sub B::SVp_IOK () { 0x01000000; } + sub B::SVp_NOK () { 0x02000000; } + sub B::SVp_POK () { 0x04000000; } + + $INC{'bytes.pm'} = 1; # dummy +} + + + +sub _encode_ascii { + join('', map { $_ <= 127 ? chr($_) : sprintf('\u%04x', $_) } unpack('C*', $_[0]) ); +} + + +sub _encode_latin1 { + join('', map { chr($_) } unpack('C*', $_[0]) ); +} + + +sub _decode_surrogates { # from http://homepage1.nifty.com/nomenclator/unicode/ucs_utf.htm + my $uni = 0x10000 + (hex($_[0]) - 0xD800) * 0x400 + (hex($_[1]) - 0xDC00); # from perlunicode + my $bit = unpack('B32', pack('N', $uni)); + + if ( $bit =~ /^00000000000(...)(......)(......)(......)$/ ) { + my ($w, $x, $y, $z) = ($1, $2, $3, $4); + return pack('B*', sprintf('11110%s10%s10%s10%s', $w, $x, $y, $z)); + } + else { + Carp::croak("Invalid surrogate pair"); + } +} + + +sub _decode_unicode { + my ($u) = @_; + my ($utf8bit); + + if ( $u =~ /^00([89a-f][0-9a-f])$/i ) { # 0x80-0xff + return pack( 'H2', $1 ); + } + + my $bit = unpack("B*", pack("H*", $u)); + + if ( $bit =~ /^00000(.....)(......)$/ ) { + $utf8bit = sprintf('110%s10%s', $1, $2); + } + elsif ( $bit =~ /^(....)(......)(......)$/ ) { + $utf8bit = sprintf('1110%s10%s10%s', $1, $2, $3); + } + else { + Carp::croak("Invalid escaped unicode"); + } + + return pack('B*', $utf8bit); +} + + +sub JSON::PP::incr_text { + $_[0]->{_incr_parser} ||= JSON::PP::IncrParser->new; + + if ( $_[0]->{_incr_parser}->{incr_parsing} ) { + Carp::croak("incr_text can not be called when the incremental parser already started parsing"); + } + + $_[0]->{_incr_parser}->{incr_text} = $_[1] if ( @_ > 1 ); + $_[0]->{_incr_parser}->{incr_text}; +} + + +1; +__END__ + +=pod + +=head1 NAME + +JSON::PP5005 - Helper module in using JSON::PP in Perl 5.005 + +=head1 DESCRIPTION + +JSON::PP calls internally. + +=head1 AUTHOR + +Makamaka Hannyaharamitu, E<lt>makamaka[at]cpan.orgE<gt> + + +=head1 COPYRIGHT AND LICENSE + +Copyright 2007-2012 by Makamaka Hannyaharamitu + +This library is free software; you can redistribute it and/or modify +it under the same terms as Perl itself. + +=cut + diff --git a/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Compat5006.pm b/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Compat5006.pm new file mode 100644 index 00000000000..7736fd8debc --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/JSON/backportPP/Compat5006.pm @@ -0,0 +1,173 @@ +package # This is JSON::backportPP + JSON::backportPP56; + +use 5.006; +use strict; + +my @properties; + +$JSON::PP56::VERSION = '1.08'; + +BEGIN { + + sub utf8::is_utf8 { + my $len = length $_[0]; # char length + { + use bytes; # byte length; + return $len != length $_[0]; # if !=, UTF8-flagged on. + } + } + + + sub utf8::upgrade { + ; # noop; + } + + + sub utf8::downgrade ($;$) { + return 1 unless ( utf8::is_utf8( $_[0] ) ); + + if ( _is_valid_utf8( $_[0] ) ) { + my $downgrade; + for my $c ( unpack( "U*", $_[0] ) ) { + if ( $c < 256 ) { + $downgrade .= pack("C", $c); + } + else { + $downgrade .= pack("U", $c); + } + } + $_[0] = $downgrade; + return 1; + } + else { + Carp::croak("Wide character in subroutine entry") unless ( $_[1] ); + 0; + } + } + + + sub utf8::encode ($) { # UTF8 flag off + if ( utf8::is_utf8( $_[0] ) ) { + $_[0] = pack( "C*", unpack( "C*", $_[0] ) ); + } + else { + $_[0] = pack( "U*", unpack( "C*", $_[0] ) ); + $_[0] = pack( "C*", unpack( "C*", $_[0] ) ); + } + } + + + sub utf8::decode ($) { # UTF8 flag on + if ( _is_valid_utf8( $_[0] ) ) { + utf8::downgrade( $_[0] ); + $_[0] = pack( "U*", unpack( "U*", $_[0] ) ); + } + } + + + *JSON::PP::JSON_PP_encode_ascii = \&_encode_ascii; + *JSON::PP::JSON_PP_encode_latin1 = \&_encode_latin1; + *JSON::PP::JSON_PP_decode_surrogates = \&JSON::PP::_decode_surrogates; + *JSON::PP::JSON_PP_decode_unicode = \&JSON::PP::_decode_unicode; + + unless ( defined &B::SVp_NOK ) { # missing in B module. + eval q{ sub B::SVp_NOK () { 0x02000000; } }; + } + +} + + + +sub _encode_ascii { + join('', + map { + $_ <= 127 ? + chr($_) : + $_ <= 65535 ? + sprintf('\u%04x', $_) : sprintf('\u%x\u%x', JSON::PP::_encode_surrogates($_)); + } _unpack_emu($_[0]) + ); +} + + +sub _encode_latin1 { + join('', + map { + $_ <= 255 ? + chr($_) : + $_ <= 65535 ? + sprintf('\u%04x', $_) : sprintf('\u%x\u%x', JSON::PP::_encode_surrogates($_)); + } _unpack_emu($_[0]) + ); +} + + +sub _unpack_emu { # for Perl 5.6 unpack warnings + return !utf8::is_utf8($_[0]) ? unpack('C*', $_[0]) + : _is_valid_utf8($_[0]) ? unpack('U*', $_[0]) + : unpack('C*', $_[0]); +} + + +sub _is_valid_utf8 { + my $str = $_[0]; + my $is_utf8; + + while ($str =~ /(?: + ( + [\x00-\x7F] + |[\xC2-\xDF][\x80-\xBF] + |[\xE0][\xA0-\xBF][\x80-\xBF] + |[\xE1-\xEC][\x80-\xBF][\x80-\xBF] + |[\xED][\x80-\x9F][\x80-\xBF] + |[\xEE-\xEF][\x80-\xBF][\x80-\xBF] + |[\xF0][\x90-\xBF][\x80-\xBF][\x80-\xBF] + |[\xF1-\xF3][\x80-\xBF][\x80-\xBF][\x80-\xBF] + |[\xF4][\x80-\x8F][\x80-\xBF][\x80-\xBF] + ) + | (.) + )/xg) + { + if (defined $1) { + $is_utf8 = 1 if (!defined $is_utf8); + } + else { + $is_utf8 = 0 if (!defined $is_utf8); + if ($is_utf8) { # eventually, not utf8 + return; + } + } + } + + return $is_utf8; +} + + +1; +__END__ + +=pod + +=head1 NAME + +JSON::PP56 - Helper module in using JSON::PP in Perl 5.6 + +=head1 DESCRIPTION + +JSON::PP calls internally. + +=head1 AUTHOR + +Makamaka Hannyaharamitu, E<lt>makamaka[at]cpan.orgE<gt> + + +=head1 COPYRIGHT AND LICENSE + +Copyright 2007-2012 by Makamaka Hannyaharamitu + +This library is free software; you can redistribute it and/or modify +it under the same terms as Perl itself. + +=cut + diff --git a/Master/tlpkg/tlperl/site/lib/Types/Serialiser.pm b/Master/tlpkg/tlperl/site/lib/Types/Serialiser.pm new file mode 100644 index 00000000000..7faa1d9b0bd --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/Types/Serialiser.pm @@ -0,0 +1,254 @@ +=head1 NAME + +Types::Serialiser - simple data types for common serialisation formats + +=encoding utf-8 + +=head1 SYNOPSIS + +=head1 DESCRIPTION + +This module provides some extra datatypes that are used by common +serialisation formats such as JSON or CBOR. The idea is to have a +repository of simple/small constants and containers that can be shared by +different implementations so they become interoperable between each other. + +=cut + +package Types::Serialiser; + +use common::sense; # required to suppress annoying warnings + +our $VERSION = '1.0'; + +=head1 SIMPLE SCALAR CONSTANTS + +Simple scalar constants are values that are overloaded to act like simple +Perl values, but have (class) type to differentiate them from normal Perl +scalars. This is necessary because these have different representations in +the serialisation formats. + +=head2 BOOLEANS (Types::Serialiser::Boolean class) + +This type has only two instances, true and false. A natural representation +for these in Perl is C<1> and C<0>, but serialisation formats need to be +able to differentiate between them and mere numbers. + +=over 4 + +=item $Types::Serialiser::true, Types::Serialiser::true + +This value represents the "true" value. In most contexts is acts like +the number C<1>. It is up to you whether you use the variable form +(C<$Types::Serialiser::true>) or the constant form (C<Types::Serialiser::true>). + +The constant is represented as a reference to a scalar containing C<1> - +implementations are allowed to directly test for this. + +=item $Types::Serialiser::false, Types::Serialiser::false + +This value represents the "false" value. In most contexts is acts like +the number C<0>. It is up to you whether you use the variable form +(C<$Types::Serialiser::false>) or the constant form (C<Types::Serialiser::false>). + +The constant is represented as a reference to a scalar containing C<0> - +implementations are allowed to directly test for this. + +=item $is_bool = Types::Serialiser::is_bool $value + +Returns true iff the C<$value> is either C<$Types::Serialiser::true> or +C<$Types::Serialiser::false>. + +For example, you could differentiate between a perl true value and a +C<Types::Serialiser::true> by using this: + + $value && Types::Serialiser::is_bool $value + +=item $is_true = Types::Serialiser::is_true $value + +Returns true iff C<$value> is C<$Types::Serialiser::true>. + +=item $is_false = Types::Serialiser::is_false $value + +Returns false iff C<$value> is C<$Types::Serialiser::false>. + +=back + +=head2 ERROR (Types::Serialiser::Error class) + +This class has only a single instance, C<error>. It is used to signal +an encoding or decoding error. In CBOR for example, and object that +couldn't be encoded will be represented by a CBOR undefined value, which +is represented by the error value in Perl. + +=over 4 + +=item $Types::Serialiser::error, Types::Serialiser::error + +This value represents the "error" value. Accessing values of this type +will throw an exception. + +The constant is represented as a reference to a scalar containing C<undef> +- implementations are allowed to directly test for this. + +=item $is_error = Types::Serialiser::is_error $value + +Returns false iff C<$value> is C<$Types::Serialiser::error>. + +=back + +=cut + +BEGIN { + # for historical reasons, and to avoid extra dependencies in JSON::PP, + # we alias *Types::Serialiser::Boolean with JSON::PP::Boolean. + package JSON::PP::Boolean; + + *Types::Serialiser::Boolean:: = *JSON::PP::Boolean::; +} + +{ + # this must done before blessing to work around bugs + # in perl < 5.18 (it seems to be fixed in 5.18). + package Types::Serialiser::BooleanBase; + + use overload + "0+" => sub { ${$_[0]} }, + "++" => sub { $_[0] = ${$_[0]} + 1 }, + "--" => sub { $_[0] = ${$_[0]} - 1 }, + fallback => 1; + + @Types::Serialiser::Boolean::ISA = Types::Serialiser::BooleanBase::; +} + +our $true = do { bless \(my $dummy = 1), Types::Serialiser::Boolean:: }; +our $false = do { bless \(my $dummy = 0), Types::Serialiser::Boolean:: }; +our $error = do { bless \(my $dummy ), Types::Serialiser::Error:: }; + +sub true () { $true } +sub false () { $false } +sub error () { $error } + +sub is_bool ($) { UNIVERSAL::isa $_[0], Types::Serialiser::Boolean:: } +sub is_true ($) { $_[0] && UNIVERSAL::isa $_[0], Types::Serialiser::Boolean:: } +sub is_false ($) { !$_[0] && UNIVERSAL::isa $_[0], Types::Serialiser::Boolean:: } +sub is_error ($) { UNIVERSAL::isa $_[0], Types::Serialiser::Error:: } + +package Types::Serialiser::Error; + +sub error { + require Carp; + Carp::croak ("caught attempt to use the Types::Serialiser::error value"); +}; + +use overload + "0+" => \&error, + "++" => \&error, + "--" => \&error, + fallback => 1; + +=head1 NOTES FOR XS USERS + +The recommended way to detect whether a scalar is one of these objects +is to check whether the stash is the C<Types::Serialiser::Boolean> or +C<Types::Serialiser::Error> stash, and then follow the scalar reference to +see if it's C<1> (true), C<0> (false) or C<undef> (error). + +While it is possible to use an isa test, directly comparing stash pointers +is faster and guaranteed to work. + +For historical reasons, the C<Types::Serialiser::Boolean> stash is +just an alias for C<JSON::PP::Boolean>. When printed, the classname +with usually be C<JSON::PP::Boolean>, but isa tests and stash pointer +comparison will normally work correctly (i.e. Types::Serialiser::true ISA +JSON::PP::Boolean, but also ISA Types::Serialiser::Boolean). + +=head1 A GENERIC OBJECT SERIALIATION PROTOCOL + +This section explains the object serialisation protocol used by +L<CBOR::XS>. It is meant to be generic enough to support any kind of +generic object serialiser. + +This protocol is called "the Types::Serialiser object serialisation +protocol". + +=head2 ENCODING + +When the encoder encounters an object that it cannot otherwise encode (for +example, L<CBOR::XS> can encode a few special types itself, and will first +attempt to use the special C<TO_CBOR> serialisation protocol), it will +look up the C<FREEZE> method on the object. + +Note that the C<FREEZE> method will normally be called I<during> encoding, +and I<MUST NOT> change the data structure that is being encoded in any +way, or it might cause memory corruption or worse. + +If it exists, it will call it with two arguments: the object to serialise, +and a constant string that indicates the name of the data model. For +example L<CBOR::XS> uses C<CBOR>, and the L<JSON> and L<JSON::XS> modules +(or any other JSON serialiser), would use C<JSON> as second argument. + +The C<FREEZE> method can then return zero or more values to identify the +object instance. The serialiser is then supposed to encode the class name +and all of these return values (which must be encodable in the format) +using the relevant form for Perl objects. In CBOR for example, there is a +registered tag number for encoded perl objects. + +The values that C<FREEZE> returns must be serialisable with the serialiser +that calls it. Therefore, it is recommended to use simple types such as +strings and numbers, and maybe array references and hashes (basically, the +JSON data model). You can always use a more complex format for a specific +data model by checking the second argument, the data model. + +The "data model" is not the same as the "data format" - the data model +indicates what types and kinds of return values can be returned from +C<FREEZE>. For example, in C<CBOR> it is permissible to return tagged CBOR +values, while JSON does not support these at all, so C<JSON> would be a +valid (but too limited) data model name for C<CBOR::XS>. similarly, a +serialising format that supports more or less the same data model as JSON +could use C<JSON> as data model without losing anything. + +=head2 DECODING + +When the decoder then encounters such an encoded perl object, it should +look up the C<THAW> method on the stored classname, and invoke it with the +classname, the constant string to identify the data model/data format, and +all the return values returned by C<FREEZE>. + +=head2 EXAMPLES + +See the C<OBJECT SERIALISATION> section in the L<CBOR::XS> manpage for +more details, an example implementation, and code examples. + +Here is an example C<FREEZE>/C<THAW> method pair: + + sub My::Object::FREEZE { + my ($self, $model) = @_; + + ($self->{type}, $self->{id}, $self->{variant}) + } + + sub My::Object::THAW { + my ($class, $model, $type, $id, $variant) = @_; + + $class->new (type => $type, id => $id, variant => $variant) + } + +=head1 BUGS + +The use of L<overload> makes this module much heavier than it should be +(on my system, this module: 4kB RSS, overload: 260kB RSS). + +=head1 SEE ALSO + +Currently, L<JSON::XS> and L<CBOR::XS> use these types. + +=head1 AUTHOR + + Marc Lehmann <schmorp@schmorp.de> + http://home.schmorp.de/ + +=cut + +1 + diff --git a/Master/tlpkg/tlperl/site/lib/Types/Serialiser/Error.pm b/Master/tlpkg/tlperl/site/lib/Types/Serialiser/Error.pm new file mode 100644 index 00000000000..3e88821ad39 --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/Types/Serialiser/Error.pm @@ -0,0 +1,27 @@ +=head1 NAME + +Types::Serialiser::Error - dummy module for Types::Serialiser + +=head1 SYNOPSIS + + # do not "use" yourself + +=head1 DESCRIPTION + +This module exists only to provide overload resolution for Storable and +similar modules that assume that class name equals module name. See +L<Types::Serialiser> for more info about this class. + +=cut + +use Types::Serialiser (); + +=head1 AUTHOR + + Marc Lehmann <schmorp@schmorp.de> + http://home.schmorp.de/ + +=cut + +1 + diff --git a/Master/tlpkg/tlperl/site/lib/auto/Canary/Stability/.packlist b/Master/tlpkg/tlperl/site/lib/auto/Canary/Stability/.packlist new file mode 100644 index 00000000000..2cfb51c8d4d --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/auto/Canary/Stability/.packlist @@ -0,0 +1 @@ +C:\tlperl\site\lib\Canary\Stability.pm
diff --git a/Master/tlpkg/tlperl/site/lib/auto/JSON/.packlist b/Master/tlpkg/tlperl/site/lib/auto/JSON/.packlist new file mode 100644 index 00000000000..ef56faff5aa --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/auto/JSON/.packlist @@ -0,0 +1,5 @@ +C:\tlperl\site\lib\JSON.pm
+C:\tlperl\site\lib\JSON\backportPP.pm
+C:\tlperl\site\lib\JSON\backportPP\Boolean.pm
+C:\tlperl\site\lib\JSON\backportPP\Compat5005.pm
+C:\tlperl\site\lib\JSON\backportPP\Compat5006.pm
diff --git a/Master/tlpkg/tlperl/site/lib/auto/JSON/XS/.packlist b/Master/tlpkg/tlperl/site/lib/auto/JSON/XS/.packlist new file mode 100644 index 00000000000..3b09ed2e542 --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/auto/JSON/XS/.packlist @@ -0,0 +1,5 @@ +C:\tlperl\bin\json_xs
+C:\tlperl\bin\json_xs.bat
+C:\tlperl\site\lib\JSON\XS.pm
+C:\tlperl\site\lib\JSON\XS\Boolean.pm
+C:\tlperl\site\lib\auto\JSON\XS\XS.dll
diff --git a/Master/tlpkg/tlperl/site/lib/auto/JSON/XS/XS.dll b/Master/tlpkg/tlperl/site/lib/auto/JSON/XS/XS.dll Binary files differnew file mode 100644 index 00000000000..5aa4c362230 --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/auto/JSON/XS/XS.dll diff --git a/Master/tlpkg/tlperl/site/lib/auto/Types/Serialiser/.packlist b/Master/tlpkg/tlperl/site/lib/auto/Types/Serialiser/.packlist new file mode 100644 index 00000000000..67c43ab14db --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/auto/Types/Serialiser/.packlist @@ -0,0 +1,2 @@ +C:\tlperl\site\lib\Types\Serialiser.pm
+C:\tlperl\site\lib\Types\Serialiser\Error.pm
diff --git a/Master/tlpkg/tlperl/site/lib/auto/common/sense/.packlist b/Master/tlpkg/tlperl/site/lib/auto/common/sense/.packlist new file mode 100644 index 00000000000..3ec71dd10ac --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/auto/common/sense/.packlist @@ -0,0 +1,2 @@ +C:\tlperl\site\lib\common\sense.pm
+C:\tlperl\site\lib\common\sense.pod
diff --git a/Master/tlpkg/tlperl/site/lib/common/sense.pm b/Master/tlpkg/tlperl/site/lib/common/sense.pm new file mode 100644 index 00000000000..14f18a4710b --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/common/sense.pm @@ -0,0 +1,16 @@ +package common::sense; + +our $VERSION = 3.74; + +# overload should be included + +sub import { + local $^W; # work around perl 5.16 spewing out warnings for next statement + # use warnings + ${^WARNING_BITS} ^= ${^WARNING_BITS} ^ "\x0c\x3f\x33\x00\x0f\xf0\x0f\xc0\xf0\xfc\x33\x00\x00\x00\x0c\x00\x00"; + # use strict, use utf8; use feature; + $^H |= 0x1c820fc0; + @^H{qw(feature_fc feature_unicode feature_switch feature_state feature_say feature___SUB__ feature_evalbytes)} = (1) x 7; +} + +1 diff --git a/Master/tlpkg/tlperl/site/lib/common/sense.pod b/Master/tlpkg/tlperl/site/lib/common/sense.pod new file mode 100644 index 00000000000..1521faf5ced --- /dev/null +++ b/Master/tlpkg/tlperl/site/lib/common/sense.pod @@ -0,0 +1,445 @@ +=head1 NAME + +common::sense - save a tree AND a kitten, use common::sense! + +=head1 SYNOPSIS + + use common::sense; + + # Supposed to be mostly the same, with much lower memory usage, as: + + # use utf8; + # use strict qw(vars subs); + # use feature qw(say state switch); + # use feature qw(unicode_strings unicode_eval current_sub fc evalbytes); + # no feature qw(array_base); + # no warnings; + # use warnings qw(FATAL closed threads internal debugging pack + # portable prototype inplace io pipe unpack malloc + # glob digit printf layer reserved taint closure + # semicolon); + # no warnings qw(exec newline unopened); + +=head1 DESCRIPTION + + “Nothing is more fairly distributed than common sense: no one thinks + he needs more of it than he already has.” + + – René Descartes + +This module implements some sane defaults for Perl programs, as defined by +two typical (or not so typical - use your common sense) specimens of Perl +coders. In fact, after working out details on which warnings and strict +modes to enable and make fatal, we found that we (and our code written so +far, and others) fully agree on every option, even though we never used +warnings before, so it seems this module indeed reflects a "common" sense +among some long-time Perl coders. + +The basic philosophy behind the choices made in common::sense can be +summarised as: "enforcing strict policies to catch as many bugs as +possible, while at the same time, not limiting the expressive power +available to the programmer". + +Two typical examples of how this philosophy is applied in practise is the +handling of uninitialised and malloc warnings: + +=over 4 + +=item I<uninitialised> + +C<undef> is a well-defined feature of perl, and enabling warnings for +using it rarely catches any bugs, but considerably limits you in what you +can do, so uninitialised warnings are disabled. + +=item I<malloc> + +Freeing something twice on the C level is a serious bug, usually causing +memory corruption. It often leads to side effects much later in the +program and there are no advantages to not reporting this, so malloc +warnings are fatal by default. + +=back + +Unfortunately, there is no fine-grained warning control in perl, so often +whole groups of useful warnings had to be excluded because of a single +useless warning (for example, perl puts an arbitrary limit on the length +of text you can match with some regexes before emitting a warning, making +the whole C<regexp> category useless). + +What follows is a more thorough discussion of what this module does, +and why it does it, and what the advantages (and disadvantages) of this +approach are. + +=head1 RATIONALE + +=over 4 + +=item use utf8 + +While it's not common sense to write your programs in UTF-8, it's quickly +becoming the most common encoding, is the designated future default +encoding for perl sources, and the most convenient encoding available +(you can do really nice quoting tricks...). Experience has shown that our +programs were either all pure ascii or utf-8, both of which will stay the +same. + +There are few drawbacks to enabling UTF-8 source code by default (mainly +some speed hits due to bugs in older versions of perl), so this module +enables UTF-8 source code encoding by default. + + +=item use strict qw(subs vars) + +Using C<use strict> is definitely common sense, but C<use strict +'refs'> definitely overshoots its usefulness. After almost two +decades of Perl hacking, we decided that it does more harm than being +useful. Specifically, constructs like these: + + @{ $var->[0] } + +Must be written like this (or similarly), when C<use strict 'refs'> is in +scope, and C<$var> can legally be C<undef>: + + @{ $var->[0] || [] } + +This is annoying, and doesn't shield against obvious mistakes such as +using C<"">, so one would even have to write (at least for the time +being): + + @{ defined $var->[0] ? $var->[0] : [] } + +... which nobody with a bit of common sense would consider +writing: clear code is clearly something else. + +Curiously enough, sometimes perl is not so strict, as this works even with +C<use strict> in scope: + + for (@{ $var->[0] }) { ... + +If that isn't hypocrisy! And all that from a mere program! + + +=item use feature qw(say state given ...) + +We found it annoying that we always have to enable extra features. If +something breaks because it didn't anticipate future changes, so be +it. 5.10 broke almost all our XS modules and nobody cared either (or at +least I know of nobody who really complained about gratuitous changes - +as opposed to bugs). + +Few modules that are not actively maintained work with newer versions of +Perl, regardless of use feature or not, so a new major perl release means +changes to many modules - new keywords are just the tip of the iceberg. + +If your code isn't alive, it's dead, Jim - be an active maintainer. + +But nobody forces you to use those extra features in modules meant for +older versions of perl - common::sense of course works there as well. +There is also an important other mode where having additional features by +default is useful: commandline hacks and internal use scripts: See "much +reduced typing", below. + +There is one notable exception: C<unicode_eval> is not enabled by +default. In our opinion, C<use feature> had one main effect - newer perl +versions don't value backwards compatibility and the ability to write +modules for multiple perl versions much, after all, you can use feature. + +C<unicode_eval> doesn't add a new feature, it breaks an existing function. + +=item no warnings, but a lot of new errors + +Ah, the dreaded warnings. Even worse, the horribly dreaded C<-w> +switch: Even though we don't care if other people use warnings (and +certainly there are useful ones), a lot of warnings simply go against the +spirit of Perl. + +Most prominently, the warnings related to C<undef>. There is nothing wrong +with C<undef>: it has well-defined semantics, it is useful, and spitting +out warnings you never asked for is just evil. + +The result was that every one of our modules did C<no warnings> in the +past, to avoid somebody accidentally using and forcing his bad standards +on our code. Of course, this switched off all warnings, even the useful +ones. Not a good situation. Really, the C<-w> switch should only enable +warnings for the main program only. + +Funnily enough, L<perllexwarn> explicitly mentions C<-w> (and not in a +favourable way, calling it outright "wrong"), but standard utilities, such +as L<prove>, or MakeMaker when running C<make test>, still enable them +blindly. + +For version 2 of common::sense, we finally sat down a few hours and went +through I<every single warning message>, identifying - according to +common sense - all the useful ones. + +This resulted in the rather impressive list in the SYNOPSIS. When we +weren't sure, we didn't include the warning, so the list might grow in +the future (we might have made a mistake, too, so the list might shrink +as well). + +Note the presence of C<FATAL> in the list: we do not think that the +conditions caught by these warnings are worthy of a warning, we I<insist> +that they are worthy of I<stopping> your program, I<instantly>. They are +I<bugs>! + +Therefore we consider C<common::sense> to be much stricter than C<use +warnings>, which is good if you are into strict things (we are not, +actually, but these things tend to be subjective). + +After deciding on the list, we ran the module against all of our code that +uses C<common::sense> (that is almost all of our code), and found only one +occurrence where one of them caused a problem: one of elmex's (unreleased) +modules contained: + + $fmt =~ s/([^\s\[]*)\[( [^\]]* )\]/\x0$1\x1$2\x0/xgo; + +We quickly agreed that indeed the code should be changed, even though it +happened to do the right thing when the warning was switched off. + + +=item much reduced typing + +Especially with version 2.0 of common::sense, the amount of boilerplate +code you need to add to get I<this> policy is daunting. Nobody would write +this out in throwaway scripts, commandline hacks or in quick internal-use +scripts. + +By using common::sense you get a defined set of policies (ours, but maybe +yours, too, if you accept them), and they are easy to apply to your +scripts: typing C<use common::sense;> is even shorter than C<use warnings; +use strict; use feature ...>. + +And you can immediately use the features of your installed perl, which +is more difficult in code you release, but not usually an issue for +internal-use code (downgrades of your production perl should be rare, +right?). + + +=item mucho reduced memory usage + +Just using all those pragmas mentioned in the SYNOPSIS together wastes +<blink>I<< B<776> kilobytes >></blink> of precious memory in my perl, for +I<every single perl process using our code>, which on our machines, is a +lot. In comparison, this module only uses I<< B<four> >> kilobytes (I even +had to write it out so it looks like more) of memory on the same platform. + +The money/time/effort/electricity invested in these gigabytes (probably +petabytes globally!) of wasted memory could easily save 42 trees, and a +kitten! + +Unfortunately, until everybody applies more common sense, there will still +often be modules that pull in the monster pragmas. But one can hope... + +=back + +=head1 THERE IS NO 'no common::sense'!!!! !!!! !! + +This module doesn't offer an unimport. First of all, it wastes even more +memory, second, and more importantly, who with even a bit of common sense +would want no common sense? + +=head1 STABILITY AND FUTURE VERSIONS + +Future versions might change just about everything in this module. We +might test our modules and upload new ones working with newer versions of +this module, and leave you standing in the rain because we didn't tell +you. In fact, we did so when switching from 1.0 to 2.0, which enabled gobs +of warnings, and made them FATAL on top. + +Maybe we will load some nifty modules that try to emulate C<say> or so +with perls older than 5.10 (this module, of course, should work with older +perl versions - supporting 5.8 for example is just common sense at this +time. Maybe not in the future, but of course you can trust our common +sense to be consistent with, uhm, our opinion). + +=head1 WHAT OTHER PEOPLE HAD TO SAY ABOUT THIS MODULE + +apeiron + + "... wow" + "I hope common::sense is a joke." + +crab + + "i wonder how it would be if joerg schilling wrote perl modules." + +Adam Kennedy + + "Very interesting, efficient, and potentially something I'd use all the time." + [...] + "So no common::sense for me, alas." + +H.Merijn Brand + + "Just one more reason to drop JSON::XS from my distribution list" + +Pista Palo + + "Something in short supply these days..." + +Steffen Schwigon + + "This module is quite for sure *not* just a repetition of all the other + 'use strict, use warnings'-approaches, and it's also not the opposite. + [...] And for its chosen middle-way it's also not the worst name ever. + And everything is documented." + +BKB + + "[Deleted - thanks to Steffen Schwigon for pointing out this review was + in error.]" + +Somni + + "the arrogance of the guy" + "I swear he tacked somenoe else's name onto the module + just so he could use the royal 'we' in the documentation" + +Anonymous Monk + + "You just gotta love this thing, its got META.json!!!" + +dngor + + "Heh. '"<elmex at ta-sa.org>"' The quotes are semantic + distancing from that e-mail address." + +Jerad Pierce + + "Awful name (not a proper pragma), and the SYNOPSIS doesn't tell you + anything either. Nor is it clear what features have to do with "common + sense" or discipline." + +acme + + "THERE IS NO 'no common::sense'!!!! !!!! !!" + +apeiron (meta-comment about us commenting^Wquoting his comment) + + "How about quoting this: get a clue, you fucktarded amoeba." + +quanth + + "common sense is beautiful, json::xs is fast, Anyevent, EV are fast and + furious. I love mlehmannware ;)" + +apeiron + + "... it's mlehmann's view of what common sense is. His view of common + sense is certainly uncommon, insofar as anyone with a clue disagrees + with him." + +apeiron (another meta-comment) + + "apeiron wonders if his little informant is here to steal more quotes" + +ew73 + + "... I never got past the SYNOPSIS before calling it shit." + [...] + How come no one ever quotes me. :(" + +chip (not willing to explain his cryptic questions about links in Changes files) + + "I'm willing to ask the question I've asked. I'm not willing to go + through the whole dance you apparently have choreographed. Either + answer the completely obvious question, or tell me to fuck off again." + +=head1 FREQUENTLY ASKED QUESTIONS + +Or frequently-come-up confusions. + +=over 4 + +=item Is this module meant to be serious? + +Yes, we would have put it under the C<Acme::> namespace otherwise. + +=item But the manpage is written in a funny/stupid/... way? + +This was meant to make it clear that our common sense is a subjective +thing and other people can use their own notions, taking the steam out +of anybody who might be offended (as some people are always offended no +matter what you do). + +This was a failure. + +But we hope the manpage still is somewhat entertaining even though it +explains boring rationale. + +=item Why do you impose your conventions on my code? + +For some reason people keep thinking that C<common::sense> imposes +process-wide limits, even though the SYNOPSIS makes it clear that it works +like other similar modules - i.e. only within the scope that C<use>s them. + +So, no, we don't - nobody is forced to use this module, and using a module +that relies on common::sense does not impose anything on you. + +=item Why do you think only your notion of common::sense is valid? + +Well, we don't, and have clearly written this in the documentation to +every single release. We were just faster than anybody else w.r.t. to +grabbing the namespace. + +=item But everybody knows that you have to use strict and use warnings, +why do you disable them? + +Well, we don't do this either - we selectively disagree with the +usefulness of some warnings over others. This module is aimed at +experienced Perl programmers, not people migrating from other languages +who might be surprised about stuff such as C<undef>. On the other hand, +this does not exclude the usefulness of this module for total newbies, due +to its strictness in enforcing policy, while at the same time not limiting +the expressive power of perl. + +This module is considerably I<more> strict than the canonical C<use +strict; use warnings>, as it makes all its warnings fatal in nature, so +you can not get away with as many things as with the canonical approach. + +This was not implemented in version 1.0 because of the daunting number +of warning categories and the difficulty in getting exactly the set of +warnings you wish (i.e. look at the SYNOPSIS in how complicated it is to +get a specific set of warnings - it is not reasonable to put this into +every module, the maintenance effort would be enormous). + +=item But many modules C<use strict> or C<use warnings>, so the memory +savings do not apply? + +I suddenly feel sad... + +But yes, that's true. Fortunately C<common::sense> still uses only a +miniscule amount of RAM. + +=item But it adds another dependency to your modules! + +It's a fact, yeah. But it's trivial to install, most popular modules have +many more dependencies. And we consider dependencies a good thing - it +leads to better APIs, more thought about interworking of modules and so +on. + +=item Why do you use JSON and not YAML for your META.yml? + +This is not true - YAML supports a large subset of JSON, and this subset +is what META.yml is written in, so it would be correct to say "the +META.yml is written in a common subset of YAML and JSON". + +The META.yml follows the YAML, JSON and META.yml specifications, and is +correctly parsed by CPAN, so if you have trouble with it, the problem is +likely on your side. + +=item But! But! + +Yeah, we know. + +=back + +=head1 AUTHOR + + Marc Lehmann <schmorp@schmorp.de> + http://home.schmorp.de/ + + Robin Redeker, "<elmex at ta-sa.org>". + +=cut + |