Semantic Search with Pinecone and OpenAI

OpenAI Logo
James Briggs
Mar 24, 2023
Open in Github

In this guide you will learn how to use the OpenAI Embedding API to generate language embeddings, and then index those embeddings in the Pinecone vector database for fast and scalable vector search.

This is a powerful and common combination for building semantic search, question-answering, threat-detection, and other applications that rely on NLP and search over a large corpus of text data.

The basic workflow looks like this:

Embed and index

  • Use the OpenAI Embedding API to generate vector embeddings of your documents (or any text data).
  • Upload those vector embeddings into Pinecone, which can store and index millions/billions of these vector embeddings, and search through them at ultra-low latencies.

Search

  • Pass your query text or document through the OpenAI Embedding API again.
  • Take the resulting vector embedding and send it as a query to Pinecone.
  • Get back semantically similar documents, even if they don't share any keywords with the query.

Architecture overview

Let's get started...

We first need to setup our environment and retrieve API keys for OpenAI and Pinecone. Let's start with our environment, we need HuggingFace Datasets for our data, and the OpenAI and Pinecone clients:

!pip install -qU pinecone-client openai datasets
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Collecting pinecone-client
  Downloading pinecone_client-2.0.13-py3-none-any.whl (175 kB)
     |████████████████████████████████| 175 kB 4.9 MB/s 
[?25hCollecting openai
  Downloading openai-0.25.0.tar.gz (44 kB)
     |████████████████████████████████| 44 kB 2.1 MB/s 
[?25h  Installing build dependencies ... [?25l[?25hdone
  Getting requirements to build wheel ... [?25l[?25hdone
    Preparing wheel metadata ... [?25l[?25hdone
Collecting datasets
  Downloading datasets-2.8.0-py3-none-any.whl (452 kB)
     |████████████████████████████████| 452 kB 59.5 MB/s 
[?25hRequirement already satisfied: urllib3>=1.21.1 in /usr/local/lib/python3.8/dist-packages (from pinecone-client) (1.24.3)
Requirement already satisfied: python-dateutil>=2.5.3 in /usr/local/lib/python3.8/dist-packages (from pinecone-client) (2.8.2)
Collecting loguru>=0.5.0
  Downloading loguru-0.6.0-py3-none-any.whl (58 kB)
     |████████████████████████████████| 58 kB 4.9 MB/s 
[?25hRequirement already satisfied: requests>=2.19.0 in /usr/local/lib/python3.8/dist-packages (from pinecone-client) (2.23.0)
Requirement already satisfied: dnspython>=2.0.0 in /usr/local/lib/python3.8/dist-packages (from pinecone-client) (2.2.1)
Requirement already satisfied: pyyaml>=5.4 in /usr/local/lib/python3.8/dist-packages (from pinecone-client) (6.0)
Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.8/dist-packages (from pinecone-client) (4.4.0)
Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.8/dist-packages (from python-dateutil>=2.5.3->pinecone-client) (1.15.0)
Requirement already satisfied: certifi>=2017.4.17 in /usr/local/lib/python3.8/dist-packages (from requests>=2.19.0->pinecone-client) (2022.12.7)
Requirement already satisfied: idna<3,>=2.5 in /usr/local/lib/python3.8/dist-packages (from requests>=2.19.0->pinecone-client) (2.10)
Requirement already satisfied: chardet<4,>=3.0.2 in /usr/local/lib/python3.8/dist-packages (from requests>=2.19.0->pinecone-client) (3.0.4)
Requirement already satisfied: numpy in /usr/local/lib/python3.8/dist-packages (from openai) (1.21.6)
Requirement already satisfied: pandas>=1.2.3 in /usr/local/lib/python3.8/dist-packages (from openai) (1.3.5)
Requirement already satisfied: openpyxl>=3.0.7 in /usr/local/lib/python3.8/dist-packages (from openai) (3.0.10)
Collecting pandas-stubs>=1.1.0.11
  Downloading pandas_stubs-1.5.2.221213-py3-none-any.whl (147 kB)
     |████████████████████████████████| 147 kB 60.7 MB/s 
[?25hRequirement already satisfied: tqdm in /usr/local/lib/python3.8/dist-packages (from openai) (4.64.1)
Requirement already satisfied: et-xmlfile in /usr/local/lib/python3.8/dist-packages (from openpyxl>=3.0.7->openai) (1.1.0)
Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.8/dist-packages (from pandas>=1.2.3->openai) (2022.6)
Collecting types-pytz>=2022.1.1
  Downloading types_pytz-2022.7.0.0-py3-none-any.whl (4.7 kB)
Collecting responses<0.19
  Downloading responses-0.18.0-py3-none-any.whl (38 kB)
Requirement already satisfied: pyarrow>=6.0.0 in /usr/local/lib/python3.8/dist-packages (from datasets) (9.0.0)
Requirement already satisfied: dill<0.3.7 in /usr/local/lib/python3.8/dist-packages (from datasets) (0.3.6)
Requirement already satisfied: packaging in /usr/local/lib/python3.8/dist-packages (from datasets) (21.3)
Requirement already satisfied: aiohttp in /usr/local/lib/python3.8/dist-packages (from datasets) (3.8.3)
Collecting huggingface-hub<1.0.0,>=0.2.0
  Downloading huggingface_hub-0.11.1-py3-none-any.whl (182 kB)
     |████████████████████████████████| 182 kB 44.4 MB/s 
[?25hRequirement already satisfied: fsspec[http]>=2021.11.1 in /usr/local/lib/python3.8/dist-packages (from datasets) (2022.11.0)
Collecting multiprocess
  Downloading multiprocess-0.70.14-py38-none-any.whl (132 kB)
     |████████████████████████████████| 132 kB 66.5 MB/s 
[?25hCollecting xxhash
  Downloading xxhash-3.1.0-cp38-cp38-manylinux_2_17_x86_64.manylinux2014_x86_64.whl (212 kB)
     |████████████████████████████████| 212 kB 70.9 MB/s 
[?25hRequirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (22.1.0)
Requirement already satisfied: aiosignal>=1.1.2 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.1)
Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (2.1.1)
Requirement already satisfied: yarl<2.0,>=1.0 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.8.2)
Requirement already satisfied: frozenlist>=1.1.1 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (1.3.3)
Requirement already satisfied: multidict<7.0,>=4.5 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (6.0.3)
Requirement already satisfied: async-timeout<5.0,>=4.0.0a3 in /usr/local/lib/python3.8/dist-packages (from aiohttp->datasets) (4.0.2)
Requirement already satisfied: filelock in /usr/local/lib/python3.8/dist-packages (from huggingface-hub<1.0.0,>=0.2.0->datasets) (3.8.2)
Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.8/dist-packages (from packaging->datasets) (3.0.9)
Collecting urllib3>=1.21.1
  Downloading urllib3-1.25.11-py2.py3-none-any.whl (127 kB)
     |████████████████████████████████| 127 kB 45.2 MB/s 
[?25hBuilding wheels for collected packages: openai
  Building wheel for openai (PEP 517) ... [?25l[?25hdone
  Created wheel for openai: filename=openai-0.25.0-py3-none-any.whl size=55880 sha256=f688888b29b707528c920841652705ab00a881bb4221ec48325e82d2b59d05a5
  Stored in directory: /root/.cache/pip/wheels/4b/92/33/6f57c7aae0b16875267999a50570e81f15eecec577ebe05a2e
Successfully built openai
Installing collected packages: urllib3, types-pytz, xxhash, responses, pandas-stubs, multiprocess, loguru, huggingface-hub, pinecone-client, openai, datasets
  Attempting uninstall: urllib3
    Found existing installation: urllib3 1.24.3
    Uninstalling urllib3-1.24.3:
      Successfully uninstalled urllib3-1.24.3
Successfully installed datasets-2.8.0 huggingface-hub-0.11.1 loguru-0.6.0 multiprocess-0.70.14 openai-0.25.0 pandas-stubs-1.5.2.221213 pinecone-client-2.0.13 responses-0.18.0 types-pytz-2022.7.0.0 urllib3-1.25.11 xxhash-3.1.0

import openai

openai.api_key = "OPENAI_API_KEY"
# get API key from top-right dropdown on OpenAI website

openai.Engine.list()  # check we have authenticated
<OpenAIObject list at 0x7f98f1d704a0> JSON: {
  "data": [
    {
      "created": null,
      "id": "babbage",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "ada",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "davinci",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "babbage-code-search-code",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-similarity-babbage-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-davinci-003",
      "object": "engine",
      "owner": "openai-internal",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-davinci-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "curie-instruct-beta",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "babbage-code-search-text",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "babbage-similarity",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "curie-search-query",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "code-search-babbage-text-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "code-cushman-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "code-search-babbage-code-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-ada-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-similarity-ada-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-davinci-insert-002",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-embedding-ada-002",
      "object": "engine",
      "owner": "openai-internal",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "ada-code-search-code",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "ada-similarity",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "code-search-ada-text-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-ada-query-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-curie-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-davinci-edit-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "davinci-search-document",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "ada-code-search-text",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-ada-doc-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "code-davinci-edit-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "davinci-instruct-beta",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-babbage-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-similarity-curie-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "code-search-ada-code-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "ada-search-query",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-davinci-query-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "curie-similarity",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "davinci-search-query",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-davinci-insert-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "babbage-search-document",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "ada-search-document",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "curie",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-babbage-doc-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-davinci-002",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-curie-doc-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-curie-query-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "babbage-search-query",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-davinci-doc-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "code-davinci-002",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-search-babbage-query-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "curie-search-document",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "text-similarity-davinci-001",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "audio-transcribe-001",
      "object": "engine",
      "owner": "openai",
      "permissions": null,
      "ready": true
    },
    {
      "created": null,
      "id": "davinci-similarity",
      "object": "engine",
      "owner": "openai-dev",
      "permissions": null,
      "ready": true
    }
  ],
  "object": "list"
}

We can now create embeddings with the OpenAI Ada similarity model like so:

MODEL = "text-embedding-ada-002"

res = openai.Embedding.create(
    input=[
        "Sample document text goes here",
        "there will be several phrases in each batch"
    ], engine=MODEL
)
res
<OpenAIObject list at 0x7f98efba5130> JSON: {
  "data": [
    {
      "embedding": [
        -0.003040769835934043,
        0.011684642173349857,
        -0.005026957020163536,
        -0.027237210422754288,
        -0.016361193731427193,
        0.03234503045678139,
        -0.016159038990736008,
        -0.001036894042044878,
        -0.025822116062045097,
        -0.00666779326274991,
        0.02014825865626335,
        0.016657691448926926,
        -0.009164425544440746,
        0.023423193022608757,
        -0.0101212989538908,
        0.01344340294599533,
        0.02522912435233593,
        -0.016873324289917946,
        0.012115909717977047,
        -0.016361193731427193,
        -0.00426887022331357,
        -0.006502698641270399,
        -0.004369948524981737,
        0.020808637142181396,
        -0.01053908932954073,
        -0.003652293002232909,
        0.01369272917509079,
        -0.026361199095845222,
        -0.0003171329153701663,
        -0.0022186669521033764,
        0.005822105333209038,
        -0.010087606497108936,
        -0.028221039101481438,
        -0.016159038990736008,
        -0.0042183310724794865,
        0.007466311100870371,
        -0.0029228453058749437,
        -0.031455542892217636,
        0.023881414905190468,
        -0.03328842669725418,
        -0.0003649345017038286,
        0.013072783127427101,
        0.00707547552883625,
        -0.005680595990270376,
        0.003106470685452223,
        -0.02981133572757244,
        0.026293814182281494,
        -0.0046428595669567585,
        0.006607146468013525,
        0.017493270337581635,
        0.027331551536917686,
        0.01578168012201786,
        -0.022196777164936066,
        0.0028537751641124487,
        -0.006489221937954426,
        0.00639151269569993,
        -0.01698114164173603,
        0.020067395642399788,
        0.0034838293213397264,
        -0.003507414134219289,
        -0.001061321352608502,
        0.0035781690385192633,
        -0.0026583571452647448,
        -0.012473052367568016,
        -0.018072785809636116,
        -0.03520217537879944,
        -0.03172508254647255,
        0.007574127987027168,
        0.00706199835985899,
        0.005582887213677168,
        0.02067386545240879,
        0.011010787449777126,
        -0.011590302921831608,
        -0.01738545298576355,
        0.00665431609377265,
        0.010181945748627186,
        -0.012075478211045265,
        -0.012419143691658974,
        0.0002756487228907645,
        -0.0010975409531965852,
        0.004612536169588566,
        -0.026603788137435913,
        -0.002378707518801093,
        0.032048534601926804,
        0.00907682441174984,
        0.014083565212786198,
        0.018261464312672615,
        0.029029665514826775,
        -0.02107817865908146,
        -0.017277637496590614,
        -0.005960245616734028,
        0.015822110697627068,
        -0.004326147958636284,
        0.016994617879390717,
        -0.013234508223831654,
        0.022008098661899567,
        -0.005380730610340834,
        0.02433963492512703,
        0.0037938025780022144,
        -0.04102427884936333,
        0.008456877432763577,
        0.009885449893772602,
        -0.01210917066782713,
        -0.010256069712340832,
        -0.03935312107205391,
        0.005710919387638569,
        0.016226423904299736,
        -0.014824805781245232,
        0.01609165221452713,
        0.014407015405595303,
        -0.021401628851890564,
        0.012628039345145226,
        -0.004541781265288591,
        -0.03326147422194481,
        0.014218335971236229,
        0.004636120982468128,
        -0.004423856735229492,
        -0.021927235648036003,
        0.0011666110949590802,
        -0.005215635988861322,
        -0.0133086321875453,
        0.02609165757894516,
        0.04000002145767212,
        -0.016442056745290756,
        0.014029656536877155,
        0.023679258301854134,
        -0.005117927212268114,
        -0.031859856098890305,
        0.0127695482224226,
        -0.005963615141808987,
        0.018099740147590637,
        0.042479805648326874,
        0.024258773773908615,
        0.02055257186293602,
        -0.02824799343943596,
        0.01880054920911789,
        -0.04541781544685364,
        0.01913747563958168,
        -0.021832896396517754,
        -0.01747979409992695,
        0.00892183743417263,
        0.03493263199925423,
        -0.0062028332613408566,
        -0.0062702191062271595,
        0.006856472697108984,
        0.011192727833986282,
        0.0006890165386721492,
        -0.021051224321126938,
        0.010862539522349834,
        -0.013840978033840656,
        0.010849062353372574,
        0.002424192614853382,
        -0.007082214113324881,
        0.012021569535136223,
        0.0036792471073567867,
        0.026239905506372452,
        0.012432620860636234,
        -0.0038712958339601755,
        -0.011327499523758888,
        -0.0205256175249815,
        0.008119950070977211,
        0.014528309926390648,
        0.019393540918827057,
        -0.010040436871349812,
        0.010424533858895302,
        0.04137468338012695,
        0.002656672615557909,
        -0.010761461220681667,
        0.00036177580477669835,
        -0.01762804202735424,
        -0.024474406614899635,
        0.016940709203481674,
        -0.04568735510110855,
        0.004164422862231731,
        -0.03309974819421768,
        0.024622654542326927,
        0.014932622201740742,
        0.014986530877649784,
        -0.00907682441174984,
        -0.02316712774336338,
        -0.01978437788784504,
        0.008018871769309044,
        0.01183289010077715,
        0.03725069388747215,
        -0.0365498848259449,
        -0.003682616399601102,
        -0.015606476925313473,
        0.0004885447560809553,
        0.002633087569847703,
        0.0006751182954758406,
        0.008018871769309044,
        0.010417794808745384,
        0.004989895038306713,
        -0.023531010374426842,
        -0.690889835357666,
        -0.02119947224855423,
        -0.003746632719412446,
        -0.011731811799108982,
        0.025714298710227013,
        0.020660387352108955,
        0.011711596511304379,
        0.010168468579649925,
        -0.010404318571090698,
        0.01413747388869524,
        -0.02314017340540886,
        0.005343668628484011,
        -0.01093666348606348,
        -0.0029683304019272327,
        -0.017574133351445198,
        -0.01892184279859066,
        0.0001869947009254247,
        0.003780325409024954,
        -0.0191644299775362,
        0.007863885723054409,
        0.00020205114560667425,
        0.013497311621904373,
        -0.028975756838917732,
        0.01375337690114975,
        0.01249326765537262,
        -0.0008625341579318047,
        0.004986525513231754,
        -0.016442056745290756,
        0.002911052666604519,
        0.024326158687472343,
        -0.037493281066417694,
        -0.011111865751445293,
        -0.005070757586508989,
        0.022466318681836128,
        0.053800567984580994,
        -0.00015077501302585006,
        -0.00010065706010209396,
        -0.00733154034242034,
        0.01130728330463171,
        0.013221031054854393,
        -0.018315372988581657,
        -0.017466315999627113,
        0.0060950168408453465,
        -0.015916449949145317,
        -0.0014260452007874846,
        0.0016214631032198668,
        0.029946107417345047,
        0.011617256328463554,
        0.01772238127887249,
        0.02726416476070881,
        -0.005569410044699907,
        0.0010242592543363571,
        0.015552569180727005,
        -0.005997307598590851,
        0.011913753114640713,
        0.001964286668226123,
        0.036657702177762985,
        -0.024407021701335907,
        0.016509443521499634,
        0.026805944740772247,
        0.005215635988861322,
        0.022668475285172462,
        -0.014501355588436127,
        -0.017870629206299782,
        -0.005842321086674929,
        0.006758763454854488,
        -0.009946096688508987,
        0.003857818664982915,
        0.027466321364045143,
        -0.014743942767381668,
        0.011132081039249897,
        0.018867934122681618,
        -0.020916452631354332,
        -0.024326158687472343,
        0.005609841085970402,
        0.011893536895513535,
        0.013760115019977093,
        -0.00504717230796814,
        -0.011266852729022503,
        0.016657691448926926,
        0.018544483929872513,
        0.007944747805595398,
        -0.03700810670852661,
        -0.015539092011749744,
        0.018787071108818054,
        -0.0038679265417158604,
        -0.04196767881512642,
        -0.006495960522443056,
        0.0021041117142885923,
        0.009097039699554443,
        0.009959573857486248,
        0.021522922441363335,
        -0.006563345901668072,
        0.006933965720236301,
        0.005020218435674906,
        0.01147574745118618,
        -0.0029733842238783836,
        0.006526283919811249,
        0.009245287626981735,
        -0.0056401644833385944,
        0.002274259924888611,
        0.024353113025426865,
        -0.0141509510576725,
        0.006994612980633974,
        0.030566053465008736,
        0.0018547853687778115,
        -0.0020518878009170294,
        0.011226421222090721,
        0.016967663541436195,
        -0.01401618029922247,
        0.0027459582779556513,
        -0.0002371126611251384,
        -0.008605125360190868,
        -0.024110523983836174,
        -0.01695418730378151,
        -0.031455542892217636,
        0.011792458593845367,
        0.00040431288653053343,
        0.015309981070458889,
        0.009656338952481747,
        0.019757423549890518,
        0.010181945748627186,
        -0.0026920498348772526,
        0.0007361863972619176,
        0.015539092011749744,
        0.006677900906652212,
        -0.010721029713749886,
        -0.03239893913269043,
        -0.0028672521002590656,
        -0.011974399909377098,
        -0.011933968402445316,
        0.002774597145617008,
        0.04528304189443588,
        -0.0163072869181633,
        0.013497311621904373,
        0.004285716451704502,
        0.01913747563958168,
        -0.006320758257061243,
        -0.005384099669754505,
        -0.014110519550740719,
        -0.02350405603647232,
        0.014946099370718002,
        -0.006893534678965807,
        -0.011098388582468033,
        0.009770894423127174,
        -0.020296506583690643,
        -0.024663085117936134,
        -0.0022136131301522255,
        0.003948789089918137,
        -0.004474395886063576,
        -0.011226421222090721,
        -0.010518873110413551,
        -0.0234771016985178,
        0.006485852412879467,
        0.004140837583690882,
        -0.004979786928743124,
        0.006071431562304497,
        -0.028140176087617874,
        -0.010343670845031738,
        -0.013018874451518059,
        0.004083560314029455,
        0.018544483929872513,
        -0.022722383961081505,
        -0.0006540603353641927,
        -0.009225072339177132,
        -0.020714296028017998,
        -0.026266859844326973,
        0.020498663187026978,
        -0.011711596511304379,
        -0.030566053465008736,
        -0.005087603814899921,
        -0.012567391619086266,
        -0.00113460305146873,
        -0.0010453172726556659,
        0.0146900350227952,
        -0.015175210312008858,
        -0.02043127827346325,
        0.0011447108117863536,
        0.027398936450481415,
        -0.02587602473795414,
        0.011502701789140701,
        0.014528309926390648,
        0.016940709203481674,
        0.01590297371149063,
        0.021320765838027,
        0.012985181994736195,
        0.0015776625368744135,
        0.01597035862505436,
        -0.015552569180727005,
        -0.0023669148795306683,
        0.02086254395544529,
        -0.0060208928771317005,
        -0.010390841402113438,
        0.013086260296404362,
        -0.005616579670459032,
        0.004299193620681763,
        -0.02270890772342682,
        0.017398931086063385,
        0.018867934122681618,
        0.0002486945304553956,
        0.021495968103408813,
        0.0082479827105999,
        0.014110519550740719,
        -0.010026959702372551,
        0.003158694365993142,
        -0.03398923575878143,
        -0.0035411070566624403,
        -0.025929933413863182,
        0.009433967061340809,
        0.012041785754263401,
        0.0033372659236192703,
        -0.02033693715929985,
        -0.02698114700615406,
        0.0040060668252408504,
        0.010188684798777103,
        0.020053919404745102,
        -0.0038746651262044907,
        -0.0036960935685783625,
        -0.001314859138801694,
        0.001159030245617032,
        0.008611864410340786,
        -0.006297172978520393,
        0.017924537882208824,
        -0.0069811358116567135,
        -0.007749330252408981,
        0.019460925832390785,
        0.016159038990736008,
        0.037493281066417694,
        0.011051218956708908,
        -0.014609172008931637,
        -0.0009273926843889058,
        0.0055289785377681255,
        -0.0012744278647005558,
        -0.00035609013866633177,
        0.027062008157372475,
        0.013261462561786175,
        0.0192587710916996,
        -0.016792461276054382,
        0.03894880786538124,
        -0.005946768447756767,
        -0.0163072869181633,
        0.021320765838027,
        0.021361196413636208,
        -0.013274939730763435,
        0.014433969743549824,
        -0.02199462056159973,
        0.015175210312008858,
        0.01997305639088154,
        -0.008119950070977211,
        0.00042705549276433885,
        0.0014075142098590732,
        0.01350405067205429,
        -0.008025610819458961,
        0.013557958416640759,
        0.026698127388954163,
        -0.0068598417565226555,
        -0.0015936666168272495,
        0.010080867446959019,
        0.004905662965029478,
        0.016266854479908943,
        0.01858491636812687,
        -0.013092998415231705,
        -0.0033204194623976946,
        0.004949463531374931,
        0.02086254395544529,
        -0.006832887884229422,
        -0.0023534379433840513,
        -0.004962940700352192,
        -0.017857152968645096,
        0.0005925710429437459,
        0.005033695604652166,
        -0.012850411236286163,
        0.012917796149849892,
        -0.0172911137342453,
        0.021832896396517754,
        0.008450139313936234,
        0.016253378242254257,
        0.008915099315345287,
        0.0015540776075795293,
        -0.005117927212268114,
        -0.017115911468863487,
        -0.037924546748399734,
        0.0013527635019272566,
        0.011064696125686169,
        0.005980461370199919,
        -0.0015481814043596387,
        -0.00771563732996583,
        0.016684645786881447,
        -0.014528309926390648,
        -0.0005445589195005596,
        0.0061489250510931015,
        0.005518870893865824,
        0.007836931385099888,
        -0.011266852729022503,
        0.0041071451269090176,
        0.012836934067308903,
        0.0039319428615272045,
        0.003632077481597662,
        -0.01870620995759964,
        0.010417794808745384,
        0.011266852729022503,
        -0.002100742422044277,
        -0.023989230394363403,
        -0.02424529567360878,
        0.04363883659243584,
        -0.013712945394217968,
        -0.007115906570106745,
        0.0019407018553465605,
        -0.009292458184063435,
        -0.021239902824163437,
        0.012035046704113483,
        0.0024107154458761215,
        -0.02196766622364521,
        -0.0008760112687014043,
        0.019339632242918015,
        0.0009400274720974267,
        -0.011239898391067982,
        0.015013485215604305,
        0.02575473114848137,
        0.007978441193699837,
        0.0017975076334550977,
        -0.028571443632245064,
        -0.013854454271495342,
        0.012250680476427078,
        0.060161758214235306,
        0.05129382759332657,
        0.005751350894570351,
        0.003921834751963615,
        0.008322106674313545,
        -0.005373992025852203,
        -0.013463619165122509,
        -0.013463619165122509,
        0.017398931086063385,
        -0.02119947224855423,
        -0.0061388174071908,
        -0.004686660133302212,
        0.01947440393269062,
        0.0026398261543363333,
        0.02221025340259075,
        -0.006735178641974926,
        0.010390841402113438,
        0.013800546526908875,
        -0.011529656127095222,
        -0.018261464312672615,
        0.007924532517790794,
        0.010458226315677166,
        0.0009349735337309539,
        0.008611864410340786,
        0.020754728466272354,
        -0.022560657933354378,
        0.010067391209304333,
        0.014380061067640781,
        -0.006634100340306759,
        -0.024514837190508842,
        -0.016442056745290756,
        0.01645553484559059,
        0.01685984805226326,
        0.023652303963899612,
        -0.0037938025780022144,
        0.030943412333726883,
        -0.010950140655040741,
        -0.013827500864863396,
        0.005195420701056719,
        0.0036994628608226776,
        0.00842992402613163,
        0.0060208928771317005,
        0.010970355942845345,
        0.003298519179224968,
        -0.002168127801269293,
        0.0031418479047715664,
        -0.021981144323945045,
        0.0032075487542897463,
        -0.013376018032431602,
        -0.00333052733913064,
        0.0017284376081079245,
        -0.00745283393189311,
        0.018167125061154366,
        -0.013025613501667976,
        0.017681950703263283,
        0.018557962030172348,
        0.00946092139929533,
        0.010525612160563469,
        -0.006526283919811249,
        -0.025835592299699783,
        -0.005549194291234016,
        -0.001648417324759066,
        -0.00956873781979084,
        -0.009285719133913517,
        -0.004363209940493107,
        -0.024690039455890656,
        -0.010667121969163418,
        -0.0013544481480494142,
        -0.02580863982439041,
        0.010330193676054478,
        0.0074056643061339855,
        0.002183289499953389,
        -0.014541787095367908,
        0.0033692740835249424,
        0.011192727833986282,
        0.0014353106962516904,
        -0.0035849076230078936,
        -0.004750676453113556,
        -0.010687337256968021,
        -0.00024637815658934414,
        -0.006927227135747671,
        -0.014272244647145271,
        0.01147574745118618,
        -0.02273586019873619,
        -0.0008566378965042531,
        0.015943404287099838,
        0.0005938345566391945,
        0.001143026165664196,
        0.0003407178446650505,
        0.02698114700615406,
        -0.004983156453818083,
        0.008584910072386265,
        -0.00983154121786356,
        -0.03161726891994476,
        -0.002240567235276103,
        -0.018167125061154366,
        0.006044477690011263,
        -0.012668469920754433,
        -0.004504719283431768,
        -0.020808637142181396,
        -0.010876016691327095,
        -0.0002609081566333771,
        -0.0004047340480610728,
        -0.006155663635581732,
        0.003510783426463604,
        0.0009947781218215823,
        -0.0014748997054994106,
        0.012863888405263424,
        -0.008059303276240826,
        -0.0019053244031965733,
        0.004723722115159035,
        -0.0111994668841362,
        -0.0011741919443011284,
        -0.013349063694477081,
        0.011381407268345356,
        0.01303235162049532,
        -0.006169140804558992,
        0.02498653531074524,
        0.0022911063861101866,
        -0.01543127465993166,
        0.012850411236286163,
        -0.019730469211935997,
        0.013402972370386124,
        0.024905674159526825,
        0.009319411590695381,
        0.023180605843663216,
        -0.022601090371608734,
        -0.004885447211563587,
        0.0036219696048647165,
        -0.012991921044886112,
        -0.007938009686768055,
        0.00841644685715437,
        -0.0021866587921977043,
        -0.018625346943736076,
        -0.02640163153409958,
        -0.0019524942617863417,
        0.001314016873948276,
        0.009272241964936256,
        -0.023086264729499817,
        -0.011880059726536274,
        -0.01880054920911789,
        0.005006741266697645,
        0.009548522531986237,
        -0.00372304767370224,
        -0.011246636509895325,
        -0.02587602473795414,
        -0.0032024949323385954,
        -0.002478101057931781,
        -0.01229111198335886,
        0.016576828435063362,
        0.005245959851890802,
        -0.004602428525686264,
        -0.008288414217531681,
        -0.0028436672873795033,
        0.004693398717790842,
        -0.0255390964448452,
        0.009575476869940758,
        0.010586258955299854,
        0.018544483929872513,
        0.017978446558117867,
        0.026051227003335953,
        -0.000489808211568743,
        0.015566046349704266,
        0.01196766085922718,
        -0.008935314603149891,
        -0.002633087569847703,
        0.004935986362397671,
        -0.010700814425945282,
        -0.016698122024536133,
        -0.002725742757320404,
        0.010545827448368073,
        0.011563348583877087,
        0.010747984051704407,
        -0.018881412222981453,
        0.013180599547922611,
        0.005107819568365812,
        -0.004481134470552206,
        0.003138478845357895,
        0.004780999850481749,
        -0.023005403578281403,
        -0.008342322893440723,
        0.011435315944254398,
        -0.01321429293602705,
        0.014649603515863419,
        -0.029083574190735817,
        0.0027004731819033623,
        0.05194072797894478,
        -0.005751350894570351,
        0.0065566073171794415,
        0.0046057975850999355,
        0.008672511205077171,
        -0.015714293345808983,
        -0.0017132757930085063,
        -0.02018868923187256,
        -0.00044432299910113215,
        -0.013126691803336143,
        0.016913754865527153,
        -0.021253380924463272,
        0.003473721444606781,
        0.046172529458999634,
        0.016657691448926926,
        0.003318734932690859,
        0.003387805074453354,
        0.019703514873981476,
        0.00733154034242034,
        -0.01904313638806343,
        0.010134776122868061,
        0.0024022923316806555,
        -0.003401282010599971,
        0.0005744612426497042,
        -0.03215635195374489,
        0.002868936862796545,
        -0.003049192950129509,
        -0.011239898391067982,
        -0.002371968934312463,
        0.019097045063972473,
        0.006421836093068123,
        0.005970353726297617,
        -0.03657683730125427,
        -0.013975748792290688,
        -0.0028167131822556257,
        -0.013564697466790676,
        0.05196768417954445,
        0.01762804202735424,
        -0.0020872652530670166,
        0.013881408609449863,
        -0.008281675167381763,
        -0.015471706166863441,
        0.0164959654211998,
        -0.007938009686768055,
        -0.011145558208227158,
        0.043207570910453796,
        0.031832899898290634,
        -0.007169815246015787,
        0.0027779664378613234,
        0.010464965365827084,
        0.024474406614899635,
        0.012129386886954308,
        -0.013194076716899872,
        0.010020220652222633,
        0.01765499636530876,
        -0.0013662405544891953,
        -0.021698124706745148,
        -0.009009438566863537,
        -0.052722398191690445,
        0.026940714567899704,
        0.005946768447756767,
        0.0018312004394829273,
        -0.011718334630131721,
        -0.00907682441174984,
        -0.019986532628536224,
        0.005842321086674929,
        -0.01726415939629078,
        0.019528312608599663,
        0.00864555686712265,
        -0.01202830858528614,
        -0.011563348583877087,
        0.013840978033840656,
        -0.024393543601036072,
        -0.004329517018049955,
        -0.005131404381245375,
        0.018072785809636116,
        -0.02405661717057228,
        0.004622643813490868,
        0.008625341579318047,
        0.01947440393269062,
        -0.01266173180192709,
        0.00041126200812868774,
        0.0005319241317920387,
        0.028840985149145126,
        -0.016266854479908943,
        -0.0070350440219044685,
        -0.0026752036064863205,
        -0.0008878036751411855,
        0.010006743483245373,
        -0.0047169835306704044,
        -0.004076821729540825,
        -0.005495286080986261,
        -0.011482485570013523,
        -0.014433969743549824,
        0.005623318254947662,
        0.008470354601740837,
        -0.012742593884468079,
        -0.017951492220163345,
        -0.0054009463638067245,
        -0.0211320873349905,
        -0.012055261991918087,
        -0.013153646141290665,
        0.0002680678735487163,
        -0.030134785920381546,
        -0.036630745977163315,
        0.016388148069381714,
        0.010431271977722645,
        0.013369278982281685,
        0.005997307598590851,
        -0.019865239039063454,
        0.020390845835208893,
        0.03997306525707245,
        -0.0049292477779090405,
        0.005057280417531729,
        0.0034501366317272186,
        0.004383425693958998,
        -0.022574136033654213,
        -0.0016037743771448731,
        -0.008187335915863514,
        0.001644205767661333,
        0.021307289600372314,
        -0.018140170723199844,
        -0.027681956067681313,
        -0.011839629150927067,
        -0.0005037064547650516,
        0.013638821430504322,
        0.0038376031443476677,
        -0.015512137673795223,
        0.004710244946181774,
        0.005424531176686287,
        0.0037735868245363235,
        0.0183692816644907,
        -0.025741253048181534,
        0.022695429623126984,
        -0.0009737201617099345,
        -0.006431944202631712,
        -0.001196092227473855,
        -0.023706212639808655,
        -0.00035693246172741055,
        0.004828169476240873,
        0.007520219776779413,
        0.011435315944254398,
        -0.0402965173125267,
        -0.01812669448554516,
        -0.0023618610575795174,
        0.01533693540841341,
        -0.0033692740835249424,
        -0.010336932726204395,
        -0.02184637263417244,
        -0.0010941717773675919,
        -0.009380058385431767,
        -0.0033979129511862993,
        0.018167125061154366,
        -0.014555263333022594,
        -0.007951486855745316,
        0.009885449893772602,
        -0.0044305953197181225,
        0.008557955734431744,
        -0.007580866571515799,
        0.010997310280799866,
        -0.03525608405470848,
        -0.01683289371430874,
        -0.016536395996809006,
        -0.01851752959191799,
        -0.006401620805263519,
        0.025714298710227013,
        0.00865903403609991,
        -0.0073517560958862305,
        -0.016172515228390694,
        -0.012681947089731693,
        -0.03919139504432678,
        -0.022964971140027046,
        -0.006182617973536253,
        -0.0023669148795306683,
        0.021320765838027,
        0.030593007802963257,
        -0.0015683970414102077,
        0.030539099127054214,
        -0.0010065706446766853,
        0.03323451802134514,
        -0.035363901406526566,
        -0.011172512546181679,
        0.017250683158636093,
        -0.019285723567008972,
        -0.007567389402538538,
        0.006829518359154463,
        -0.01950135827064514,
        -0.006826149299740791,
        0.03894880786538124,
        0.004555258434265852,
        -0.0036725087556988,
        0.033369291573762894,
        -0.0019440711475908756,
        -0.02762804739177227,
        -0.008086257614195347,
        0.006954181473702192,
        0.0010705868480727077,
        -0.007702160160988569,
        0.001126179820857942,
        -0.007190030533820391,
        -0.010801892727613449,
        -0.0023231145460158587,
        -0.013221031054854393,
        -0.007257416378706694,
        0.004093667957931757,
        0.010471703484654427,
        0.0019255400402471423,
        0.005801890045404434,
        -0.02707548625767231,
        -0.012985181994736195,
        -0.01849057525396347,
        -0.014757419936358929,
        0.0274124126881361,
        0.004626013338565826,
        -0.014973053708672523,
        0.03690028935670853,
        -0.00906334724277258,
        -0.0030053923837840557,
        -0.016226423904299736,
        0.023328853771090508,
        -0.016603782773017883,
        -0.012479790486395359,
        0.002633087569847703,
        -0.023315375670790672,
        -0.023450147360563278,
        -0.0031772253569215536,
        -0.005124665796756744,
        -0.021684646606445312,
        0.007277631666511297,
        -0.005043803248554468,
        -0.010134776122868061,
        -0.015768202021718025,
        -0.008793804794549942,
        0.006543130148202181,
        -0.02477090246975422,
        -0.04568735510110855,
        -0.02304583415389061,
        0.018409714102745056,
        -0.0006502699106931686,
        -0.009373320266604424,
        -0.011792458593845367,
        -0.014878714457154274,
        0.021401628851890564,
        0.010835585184395313,
        -0.011900275945663452,
        -0.007553912233561277,
        0.01993262581527233,
        0.011886798776686192,
        -0.01904313638806343,
        0.021051224321126938,
        0.22964972257614136,
        -0.00021447535254992545,
        0.012991921044886112,
        0.040754739195108414,
        0.013288416899740696,
        0.01239892840385437,
        0.014946099370718002,
        0.0015111194225028157,
        -0.010687337256968021,
        0.007574127987027168,
        0.007095691282302141,
        0.003965635318309069,
        -0.0014117257669568062,
        -0.00031439537997357547,
        0.027358505874872208,
        0.012870626524090767,
        -0.02083558961749077,
        -0.04021565616130829,
        -0.011603780090808868,
        -0.0293800700455904,
        -0.000714286114089191,
        -0.0044474415481090546,
        -0.00025290611665695906,
        -0.010619951412081718,
        0.00586927542462945,
        -0.008982484228909016,
        -0.011118603870272636,
        -0.003824125975370407,
        0.01898922771215439,
        0.015242595225572586,
        -0.01038410235196352,
        -0.013376018032431602,
        0.01037062518298626,
        0.007641513366252184,
        -0.04223721846938133,
        0.02095688506960869,
        0.014946099370718002,
        0.011482485570013523,
        0.0223989337682724,
        0.014366584829986095,
        0.0014951153425499797,
        -0.01731806807219982,
        -0.02314017340540886,
        -0.01892184279859066,
        -0.0016644214047119021,
        0.0141509510576725,
        0.007991918362677097,
        -0.01978437788784504,
        0.008483831770718098,
        0.0022995295003056526,
        -0.00984501838684082,
        0.014191382564604282,
        0.020390845835208893,
        0.012351758778095245,
        0.008955529890954494,
        -0.007223723456263542,
        0.016630737110972404,
        0.013672513887286186,
        -0.024070093408226967,
        0.010485180653631687,
        -0.008099734783172607,
        0.015795156359672546,
        -0.005117927212268114,
        0.017493270337581635,
        -0.011253375560045242,
        -0.005161727778613567,
        -0.015377366915345192,
        -0.02095688506960869,
        -0.014460924081504345,
        -0.0020872652530670166,
        0.002885783091187477,
        0.005431269761174917,
        -0.0017941384576261044,
        0.0013333902461454272,
        -0.006347712129354477,
        -0.014528309926390648,
        0.006718332413583994,
        0.02873316965997219,
        0.01769542694091797,
        0.027560660615563393,
        -0.020134782418608665,
        0.0009678239584900439,
        0.01870620995759964,
        -0.02335580810904503,
        -0.025714298710227013,
        -0.022843677550554276,
        -0.0026516185607761145,
        0.0009543468477204442,
        -0.009804587811231613,
        -0.017088957130908966,
        -0.01784367486834526,
        -0.012500006705522537,
        -0.012560653500258923,
        -0.01597035862505436,
        0.017708905041217804,
        0.0018278311472386122,
        -0.015983836725354195,
        0.016078175976872444,
        -0.006627361755818129,
        -0.0009989897953346372,
        -0.032506756484508514,
        0.03964961692690849,
        0.004019543994218111,
        0.014757419936358929,
        0.0038746651262044907,
        0.00012940117449034005,
        -0.00453167362138629,
        0.006886796094477177,
        -0.009433967061340809,
        -0.011799197643995285,
        0.019514834508299828,
        0.0063982512801885605,
        0.015673862770199776,
        -0.008254721760749817,
        -0.012365235947072506,
        0.021145563572645187,
        0.002660041907802224,
        -0.01704852655529976,
        0.002114219358190894,
        0.003965635318309069,
        -0.01990567147731781,
        -0.005303237121552229,
        -0.016805939376354218,
        0.004312670789659023,
        -0.016536395996809006,
        -0.013052567839622498,
        -0.0055289785377681255,
        -0.004821430891752243,
        0.012486529536545277,
        -0.01490566786378622,
        0.0035781690385192633,
        0.007486526854336262,
        0.006613885052502155,
        0.0034113898873329163,
        -0.007163076661527157,
        3.271880996180698e-05,
        0.0005382415256462991,
        0.011388146318495274,
        -0.019730469211935997,
        0.015471706166863441,
        0.010053914040327072,
        -0.008072780445218086,
        0.001987871713936329,
        -0.013389495201408863,
        0.0016172515461221337,
        -0.005026957020163536,
        0.009649600833654404,
        -0.01923181675374508,
        -0.0019154322799295187,
        -0.017008095979690552,
        -0.027223734185099602,
        -0.013288416899740696,
        -0.0082479827105999,
        0.008045826107263565,
        0.028814030811190605,
        -0.01671159826219082,
        -0.018962273374199867,
        -0.027250688523054123,
        0.016361193731427193,
        -0.014649603515863419,
        -0.034043144434690475,
        -0.0005424531409516931,
        0.026805944740772247,
        -0.01882750354707241,
        0.003335581161081791,
        -0.002836928702890873,
        -0.17617259919643402,
        0.0017419146606698632,
        0.01760108768939972,
        -0.022722383961081505,
        0.00880054384469986,
        -0.012405666522681713,
        0.026051227003335953,
        0.006974397227168083,
        -0.0006228945567272604,
        0.003253034083172679,
        -0.007473049685359001,
        -0.015283026732504368,
        -0.035363901406526566,
        -0.00625337241217494,
        -0.013699468225240707,
        0.0062230490148067474,
        -0.03479786217212677,
        0.010849062353372574,
        0.018975751474499702,
        0.02916443534195423,
        0.003743263427168131,
        -0.010707552544772625,
        0.01911052130162716,
        -0.03110513836145401,
        0.00852426327764988,
        -0.005704180803149939,
        -0.006927227135747671,
        0.026994623243808746,
        -0.00440701050683856,
        -0.007823454216122627,
        0.0035950152669101954,
        -0.004690029192715883,
        -0.0022388824727386236,
        0.005222374573349953,
        0.011994615197181702,
        -0.0024292466696351767,
        0.010694075375795364,
        -0.0029245298355817795,
        -0.009487875737249851,
        0.020822113379836082,
        0.014353107661008835,
        0.041212961077690125,
        0.003261457197368145,
        -0.02421834133565426,
        0.0018194079166278243,
        0.007304586004465818,
        0.01990567147731781,
        -0.00467655248939991,
        0.0017739228205755353,
        -0.013342324644327164,
        0.02230459451675415,
        -0.03436659649014473,
        -0.00612197071313858,
        -0.0025825484190136194,
        0.0001508802961325273,
        -0.0006932281539775431,
        0.0032917805947363377,
        -0.01332210935652256,
        -0.0061388174071908,
        0.00835579913109541,
        0.015835586935281754,
        -0.03528303653001785,
        0.010761461220681667,
        0.02599731832742691,
        -0.010619951412081718,
        -0.01196766085922718,
        -0.02685985155403614,
        0.013072783127427101,
        -0.00816712062805891,
        0.009669816121459007,
        -0.01846362091600895,
        -0.021671170368790627,
        -0.008281675167381763,
        -0.005902967881411314,
        0.0028790447395294905,
        -0.009973051026463509,
        -0.026145566254854202,
        0.0223989337682724,
        -0.01067386008799076,
        -0.009022915735840797,
        -0.02611861191689968,
        0.024636130779981613,
        -0.007412402890622616,
        0.019339632242918015,
        0.004366578999906778,
        0.015080870129168034,
        -0.020539093762636185,
        -0.008126689121127129,
        0.005478439386934042,
        0.013382756151258945,
        0.044717006385326385,
        -0.008712942712008953,
        -0.01580863445997238,
        -0.004053236450999975,
        0.003682616399601102,
        0.01880054920911789,
        -0.0017890845192596316,
        0.019487880170345306,
        0.003955527674406767,
        0.002309637377038598,
        -0.001693902537226677,
        -0.02067386545240879,
        -0.0095822149887681,
        0.010249331593513489,
        0.029245298355817795,
        0.014757419936358929,
        0.013814023695886135,
        -0.003611861728131771,
        0.021954189985990524,
        -0.0003476669662632048,
        -0.045768219977617264,
        0.004336255602538586,
        0.015242595225572586,
        0.01229111198335886,
        -0.005966984201222658,
        0.037762824445962906,
        -0.022843677550554276,
        -0.03347710520029068,
        0.005997307598590851,
        -0.002313006669282913,
        0.04094341769814491,
        0.007412402890622616,
        -0.0021698125638067722,
        0.011556609533727169,
        0.010229116305708885,
        -0.008497308939695358,
        -0.08361190557479858,
        -0.012654992751777172,
        0.018719686195254326,
        0.03460918366909027,
        -0.018207557499408722,
        0.015202164649963379,
        -0.011650949716567993,
        0.020714296028017998,
        0.004959571175277233,
        0.001922170864418149,
        0.0030542470049113035,
        -0.018773594871163368,
        -0.00827493704855442,
        -0.022870631888508797,
        0.024353113025426865,
        0.012358496896922588,
        -0.0006599565385840833,
        -0.005798520520329475,
        -0.0016105129616335034,
        0.02184637263417244,
        -0.009784371592104435,
        -0.002656672615557909,
        0.013369278982281685,
        -0.013295155018568039,
        0.0024511469528079033,
        -0.0059905690141022205,
        -0.027547184377908707,
        0.023247990757226944,
        0.007412402890622616,
        -0.011677904054522514,
        -0.003574799746274948,
        -0.00984501838684082,
        0.02095688506960869,
        -0.013039090670645237,
        -0.004797846078872681,
        0.017708905041217804,
        -0.04280325770378113,
        -0.009326150640845299,
        0.007082214113324881,
        -0.007681944873183966,
        0.0024646238889545202,
        0.026603788137435913,
        0.01401618029922247,
        -0.040134791284799576,
        0.0007277631666511297,
        -0.011987877078354359,
        -0.007884101010859013,
        0.03757414221763611,
        0.005256067495793104,
        -0.027277642861008644,
        -0.035552579909563065,
        0.01633424125611782,
        -0.030269557610154152,
        -0.009299196302890778,
        0.030080879107117653,
        0.016442056745290756,
        0.022345025092363358,
        -0.0016543135279789567,
        -0.00518531259149313,
        -0.007688683457672596,
        -0.0067688715644180775,
        -0.009103778749704361,
        -0.002673518843948841,
        0.03000001609325409,
        -0.004797846078872681,
        0.0054009463638067245,
        -0.01642858050763607,
        0.000922338746022433,
        -0.0012390505289658904,
        -0.013800546526908875,
        0.0040970370173454285,
        0.0022944756783545017,
        -0.011219682171940804,
        0.020929930731654167,
        -0.00790431722998619,
        -0.004420487675815821,
        -0.030107833445072174,
        -0.024433976039290428,
        0.012971704825758934,
        -0.0052796523086726665,
        -0.01731806807219982,
        -0.01628033258020878,
        -0.010498657822608948,
        0.004086929373443127,
        0.021253380924463272,
        0.02338276244699955,
        0.004103775601834059,
        0.0047338297590613365,
        0.014555263333022594,
        -0.035336945205926895,
        0.018625346943736076,
        0.04161727428436279,
        -0.008969007059931755,
        -0.021792463958263397,
        -0.0045518893748521805,
        0.006664424203336239,
        -0.0008301048655994236,
        0.00505391089245677,
        0.007237200625240803,
        -0.004066713619977236,
        -0.03954179957509041,
        -0.026226429268717766,
        -0.06517523527145386,
        0.011765505187213421,
        0.010626690462231636,
        -0.02460917830467224,
        -0.0036219696048647165,
        0.01830189675092697,
        -0.004868600983172655,
        -0.0041340989992022514,
        -0.017345022410154343,
        0.006708224769681692,
        -0.02602427266538143,
        0.00587601400911808,
        0.008787066675722599,
        -0.0011354453163221478,
        -0.026266859844326973,
        -0.0036017538513988256,
        0.02057952620089054,
        0.0018648931290954351,
        0.025377372279763222,
        0.02587602473795414,
        0.0008869614102877676,
        -0.021428583189845085,
        -0.002978438278660178,
        0.010929924435913563,
        -0.014326153323054314,
        0.005582887213677168,
        -0.02978438138961792,
        0.026913760229945183,
        -0.010283024050295353,
        -0.021159041672945023,
        -0.0020822114311158657,
        -0.0390835776925087,
        0.0035411070566624403,
        0.011024264618754387,
        -0.000971193250734359,
        -0.0025657021906226873,
        -0.004417118150740862,
        0.032938022166490555,
        0.005370622966438532,
        0.015202164649963379,
        -0.008335583843290806,
        -0.02249327301979065,
        0.006880057509988546,
        -0.02520217001438141,
        0.0024006078019738197,
        0.028382765129208565,
        -0.025862546637654305,
        -0.019959580153226852,
        0.035579532384872437,
        0.009346365928649902,
        0.01196766085922718,
        -0.003844341728836298,
        -0.028328856453299522,
        -0.02421834133565426,
        -0.008652295917272568,
        -0.008739897049963474,
        0.014420492574572563,
        0.0015035385731607676,
        0.017668472602963448,
        -0.01103100273758173,
        0.035768214613199234,
        -0.005886121653020382,
        -0.0023652303498238325,
        0.008638818748295307,
        -0.008706203661859035,
        0.009413751773536205,
        -0.028571443632245064,
        0.005579517688602209,
        -0.008510786108672619,
        -0.021617261692881584,
        -0.03517521917819977,
        -0.02328842133283615,
        -0.010485180653631687,
        0.038167137652635574,
        0.026994623243808746,
        0.016927232965826988,
        -0.008160381577908993,
        0.02230459451675415,
        0.007102429401129484,
        0.02787063457071781,
        0.01966308243572712,
        0.021334243938326836,
        -0.008052565157413483,
        0.009299196302890778,
        0.04218330979347229,
        0.01664421334862709,
        -0.0225876122713089,
        0.004349732771515846,
        -0.013450141996145248,
        0.009690032340586185,
        -0.010896231979131699,
        0.010714291594922543,
        0.013840978033840656,
        0.001228942652232945,
        -0.00852426327764988,
        -0.013908362947404385,
        4.984814836461737e-07,
        0.004363209940493107,
        0.02194071188569069,
        0.018005400896072388,
        0.005134773440659046,
        0.009535045363008976,
        -0.002845351817086339,
        -0.023544486612081528,
        -0.008969007059931755,
        -0.003072777995839715,
        -0.03668465465307236,
        -0.047088973224163055,
        -0.0006873318925499916,
        0.020849067717790604,
        -0.007863885723054409,
        -0.013362540863454342,
        0.015983836725354195,
        0.014487878419458866,
        -0.0345822274684906,
        0.016698122024536133,
        -0.013402972370386124,
        -0.010485180653631687,
        -0.03630729764699936,
        0.03129381686449051,
        -0.014649603515863419,
        -0.0017216990236192942,
        0.039433982223272324,
        -0.008969007059931755,
        0.023490577936172485,
        0.008780327625572681,
        0.024663085117936134,
        -0.01654987409710884,
        -0.0001794138370314613,
        0.008180596865713596,
        -0.0070350440219044685,
        0.018045831471681595,
        -0.012803241610527039,
        -0.026563355699181557,
        0.0026145565789192915,
        -0.01101752556860447,
        -0.01425876747816801,
        0.04463614150881767,
        -0.00372304767370224,
        0.07493265718221664,
        -0.002897575730457902,
        -0.009049870073795319,
        0.01104447990655899,
        -0.015592999756336212,
        0.02765500172972679,
        0.02138815075159073,
        0.014407015405595303,
        -0.01230458915233612,
        0.007095691282302141,
        0.0225876122713089,
        0.007129383739084005,
        -0.001028470927849412,
        -0.016725076362490654,
        -0.020849067717790604,
        -0.01249326765537262,
        -0.010020220652222633,
        0.011576825752854347,
        0.0031856484711170197,
        0.0020805266685783863,
        0.014272244647145271,
        -0.005097711458802223,
        0.021118609234690666,
        0.01654987409710884,
        -0.010350409895181656,
        -0.0025000013411045074,
        0.028167130425572395,
        -0.027237210422754288,
        -0.01092318631708622,
        -0.016603782773017883,
        -0.0021091655362397432,
        -0.003982482012361288,
        -0.022264162078499794,
        -0.025188691914081573,
        0.00041610535117797554,
        0.015417797490954399,
        -0.008234505541622639,
        -0.027237210422754288,
        -0.0056873345747590065,
        0.009016177617013454,
        0.003958896733820438,
        0.028194084763526917,
        -0.013814023695886135,
        0.002461254596710205,
        -0.03412400558590889,
        0.011145558208227158,
        -0.00935310497879982,
        -0.0006279484368860722,
        -0.01344340294599533
      ],
      "index": 0,
      "object": "embedding"
    },
    {
      "embedding": [
        -0.033118706196546555,
        -0.009973089210689068,
        0.007738340180367231,
        0.01987677626311779,
        -0.005902653094381094,
        0.016267864033579826,
        -0.007432970684021711,
        -0.015684885904192924,
        -0.034201379865407944,
        -0.01390125136822462,
        0.006142090540379286,
        0.03406257554888725,
        -0.016573233529925346,
        0.008140872232615948,
        0.007224764209240675,
        0.00477486802265048,
        0.019057830795645714,
        0.003567270701751113,
        0.0037164853420108557,
        -0.022861069068312645,
        0.00337988487444818,
        0.0012926150811836123,
        -0.010354801081120968,
        -0.013200289569795132,
        -0.025415068492293358,
        0.0065203323028981686,
        0.005437658634036779,
        -0.011819186620414257,
        -0.015351756475865841,
        0.009827344678342342,
        -0.0061247400008141994,
        0.008890415541827679,
        0.001048840000294149,
        -0.019460363313555717,
        -0.013720805756747723,
        -0.016406668350100517,
        -0.018405450507998466,
        -0.0063155959360301495,
        0.006818761583417654,
        -0.009223545901477337,
        0.03872640058398247,
        -0.0049553136341273785,
        0.0011347251711413264,
        -0.0005855806521140039,
        -0.006638315971940756,
        -0.009355410002171993,
        0.006211492698639631,
        -0.017697548493742943,
        -0.020903928205370903,
        0.016101298853754997,
        0.03545061871409416,
        -0.010292339138686657,
        -0.019932297989726067,
        -0.013970653526484966,
        -0.0018807983724400401,
        0.02616461180150509,
        0.007696698885411024,
        0.010104953311383724,
        -0.015226832590997219,
        0.0008918176754377782,
        0.018100081011652946,
        0.00840460043400526,
        -0.004760987590998411,
        0.025345666334033012,
        -0.02309703640639782,
        -0.009633018635213375,
        0.0007556159398518503,
        0.006478691007941961,
        -0.004073906224220991,
        -0.0007152759353630245,
        0.019335439428687096,
        -0.010743453167378902,
        -0.012887979857623577,
        -0.0021653471048921347,
        0.01760038547217846,
        -0.015101908706128597,
        -0.007911845110356808,
        -0.00519475108012557,
        0.008578105829656124,
        0.005111468490213156,
        0.014865941368043423,
        -0.020237667486071587,
        -0.0015129669336602092,
        0.018627537414431572,
        0.020751243457198143,
        0.016309505328536034,
        0.00021471291256602854,
        0.012901860289275646,
        -0.010035551153123379,
        -0.003027668921276927,
        0.020279308781027794,
        0.020307069644331932,
        0.023818818852305412,
        0.01589309237897396,
        -0.014727137051522732,
        0.009973089210689068,
        0.010118833743035793,
        0.05702080950140953,
        0.013783267699182034,
        -0.024138068780303,
        -0.017808591946959496,
        0.02051527611911297,
        -0.02233361266553402,
        -0.009445632807910442,
        -0.03381272777915001,
        0.005704856943339109,
        0.0023527329321950674,
        -0.0071553620509803295,
        -0.0015311850002035499,
        0.010500545613467693,
        0.00840460043400526,
        0.04022548720240593,
        0.0032393455039709806,
        -0.03414585813879967,
        -0.013637523166835308,
        0.0048408000729978085,
        0.00618720194324851,
        -0.026553263887763023,
        -0.015684885904192924,
        -0.021806156262755394,
        -0.007120660971850157,
        -0.005798549856990576,
        0.01851649396121502,
        -0.014505050145089626,
        0.009889806620776653,
        0.015407278202474117,
        -0.0226112212985754,
        -0.017683668062090874,
        -0.005014305468648672,
        0.013387675397098064,
        0.023638373240828514,
        0.026636546477675438,
        0.020459754392504692,
        -0.015795929357409477,
        -0.030009491369128227,
        0.004566661547869444,
        -0.03470107540488243,
        -0.004531960468739271,
        -0.021486906334757805,
        -0.021528547629714012,
        0.0077522206120193005,
        0.009390111081302166,
        -0.0045111398212611675,
        0.014699376188218594,
        0.006541152950376272,
        0.025165220722556114,
        0.029287708923220634,
        0.017170092090964317,
        -0.012596490792930126,
        -0.0174477007240057,
        0.01943260245025158,
        -0.013450137339532375,
        0.00994532834738493,
        -0.0068499925546348095,
        0.005284973885864019,
        -0.003095336025580764,
        0.011978811584413052,
        0.012887979857623577,
        -0.03747716173529625,
        0.008057589642703533,
        0.016892483457922935,
        0.010646290145814419,
        0.022722264751791954,
        0.00023509979655500501,
        0.013186409138143063,
        0.02095944993197918,
        0.03561718389391899,
        -0.01680920086801052,
        0.00443132733926177,
        0.0016153351170942187,
        -0.019321558997035027,
        0.0016257454408332705,
        -0.01898842863738537,
        0.01275611575692892,
        0.013651403598487377,
        0.016753679141402245,
        0.012624251656234264,
        -0.007495432626456022,
        -0.017059048637747765,
        -0.04552781209349632,
        0.00864750798791647,
        0.01680920086801052,
        0.017503222450613976,
        0.007096370216459036,
        -0.014407887123525143,
        -0.006586264353245497,
        0.004295993130654097,
        -0.011298670433461666,
        0.005101058166474104,
        0.01069487165659666,
        -0.0025349135976284742,
        0.013741626404225826,
        0.012395224533975124,
        0.008432361297309399,
        -0.6911343932151794,
        -0.015962494537234306,
        0.011402773670852184,
        -0.027608176693320274,
        0.016087418422102928,
        -0.0008892150945030153,
        0.00012676736514549702,
        -0.011999632231891155,
        -0.020376471802592278,
        0.03012053482234478,
        -0.0019901066552847624,
        0.002602580701932311,
        -0.013373794965445995,
        -0.012124556116759777,
        -0.0006745021673850715,
        -0.015851451084017754,
        0.002172287320718169,
        -0.019002309069037437,
        0.010431143455207348,
        0.00802288856357336,
        -0.02444343827664852,
        0.01399147417396307,
        -0.0004532828170340508,
        0.027288926765322685,
        -0.006919394712895155,
        -0.007047788705676794,
        0.009577496908605099,
        -0.0023232370149344206,
        -0.026983557268977165,
        -0.004153718706220388,
        -0.015559962950646877,
        0.005163520108908415,
        0.01914111338555813,
        -0.029870687052607536,
        0.04316813871264458,
        0.019793493673205376,
        -0.006867343094199896,
        0.008453181944787502,
        0.029870687052607536,
        0.019779613241553307,
        -0.02876025252044201,
        -0.0011754989391192794,
        0.005926943849772215,
        -0.003098806133493781,
        -0.010375621728599072,
        0.02876025252044201,
        0.025331785902380943,
        -5.1428080041659996e-05,
        -0.010382561944425106,
        -0.015795929357409477,
        0.0005799417267553508,
        -0.006784060504287481,
        0.012485447339713573,
        -0.0027448551263660192,
        -5.012678957427852e-05,
        0.014199680648744106,
        0.02051527611911297,
        -0.004032264929264784,
        -0.001795780728571117,
        -0.00645440025255084,
        0.012034333311021328,
        0.005899182986468077,
        0.00028086183010600507,
        -0.01833604834973812,
        -0.019085591658949852,
        0.012075974605977535,
        -0.012436865828931332,
        -0.024818209931254387,
        0.02479044906795025,
        -0.014643854461610317,
        -0.0002648125810083002,
        0.008286616764962673,
        0.010209056548774242,
        -0.013720805756747723,
        0.009286007843911648,
        -0.0032792517449706793,
        -0.00288365944288671,
        -0.01958528719842434,
        0.0077661010436713696,
        0.010118833743035793,
        0.0004801761533599347,
        -0.00014856398047413677,
        -0.04330694302916527,
        0.01116680633276701,
        0.012887979857623577,
        0.001958875684067607,
        -9.309648157795891e-05,
        -0.012853278778493404,
        -0.019154993817210197,
        -0.004528490360826254,
        -0.012027393095195293,
        -0.011583219282329082,
        0.0005166122573427856,
        -0.041197117418050766,
        0.0031491226982325315,
        -0.008807132951915264,
        -0.007606476079672575,
        0.004368865396827459,
        0.01986289583146572,
        -0.0388929657638073,
        -0.007953486405313015,
        0.009390111081302166,
        0.002942651277408004,
        0.003128302050754428,
        0.04280724748969078,
        -0.00014466010907199234,
        -0.015046386979520321,
        0.00993144791573286,
        0.011437474749982357,
        6.609470437979326e-05,
        0.012478507123887539,
        0.016934124752879143,
        -0.013477898202836514,
        -0.007856323383748531,
        -0.0015051591908559203,
        -0.039531465619802475,
        0.025803720578551292,
        -3.9662245399085805e-05,
        0.013699985109269619,
        -0.01682308129966259,
        0.03025933913886547,
        0.0218477975577116,
        0.003256696043536067,
        -0.006541152950376272,
        0.025040296837687492,
        0.006551563274115324,
        -0.005656275432556868,
        -0.006485631223767996,
        0.011021061800420284,
        0.008841834031045437,
        0.002111560432240367,
        -0.016115179285407066,
        0.021348102018237114,
        -0.0166842769831419,
        0.00979264359921217,
        0.0009421342401765287,
        0.019696330651640892,
        -0.01223559956997633,
        0.033701684325933456,
        -0.04164129123091698,
        -0.004667294677346945,
        0.012152316980063915,
        0.0004439569020178169,
        -0.0049934848211705685,
        -0.002095944946631789,
        -0.011742844246327877,
        -0.018891265615820885,
        -0.00011310382251394913,
        -0.02494313381612301,
        0.004899791907519102,
        0.012825517915189266,
        0.0014860735973343253,
        -0.009327649138867855,
        0.01345707755535841,
        -0.0033018074464052916,
        0.014976984821259975,
        0.010535246692597866,
        -0.004261292051523924,
        -0.009161083959043026,
        -0.023624492809176445,
        0.01757262460887432,
        0.02231973223388195,
        0.008661388419568539,
        -0.019515885040163994,
        -0.0033677394967526197,
        -0.03467331454157829,
        0.0035447150003165007,
        0.017003526911139488,
        -0.0011156395776197314,
        -0.025775959715247154,
        0.0036227924283593893,
        -0.036338966339826584,
        0.008085350506007671,
        0.015323995612561703,
        0.009952268563210964,
        0.026081329211592674,
        -0.024804329499602318,
        0.01958528719842434,
        0.00887653511017561,
        -0.020695721730589867,
        0.009383170865476131,
        0.03239692375063896,
        -0.008911236189305782,
        -0.003398970467969775,
        0.04591646417975426,
        0.022375253960490227,
        0.025609394535422325,
        0.023457927629351616,
        -0.008911236189305782,
        0.028177274391055107,
        0.014699376188218594,
        0.006051867734640837,
        -0.014033115468919277,
        0.006895103957504034,
        0.0038379391189664602,
        0.00603451719507575,
        -0.004552781116217375,
        0.006485631223767996,
        0.004577071871608496,
        0.005437658634036779,
        0.05257907137274742,
        0.007467671763151884,
        0.0132141700014472,
        -0.0011277849553152919,
        -0.0021861677523702383,
        -0.05840885266661644,
        0.00016374570259358734,
        -0.02280554734170437,
        0.02387434057891369,
        0.01607353799045086,
        -0.0020178675185889006,
        -0.027455491945147514,
        -0.016948005184531212,
        -0.00841848086565733,
        -0.026525503024458885,
        0.012395224533975124,
        0.0020074571948498487,
        0.003475312842056155,
        0.0008818411151878536,
        0.004858150612562895,
        -0.004223120864480734,
        -0.033590640872716904,
        0.027705339714884758,
        0.007849383167922497,
        -0.039226096123456955,
        0.004643003921955824,
        -0.0002806449483614415,
        0.02810787223279476,
        0.003709545126184821,
        0.001444432302378118,
        0.017961276695132256,
        0.025803720578551292,
        0.018932906910777092,
        0.006461340468376875,
        0.010139654390513897,
        0.007391329389065504,
        0.03620016202330589,
        -0.01757262460887432,
        0.029704121872782707,
        0.01636502705514431,
        -0.00523292226716876,
        0.022389134392142296,
        0.02781638316810131,
        -0.011597099713981152,
        0.011409713886678219,
        0.004504199605435133,
        0.02356897108256817,
        -0.006565443705767393,
        -0.012714474461972713,
        -0.001144267967902124,
        0.005944294389337301,
        0.012568729929625988,
        -0.018780222162604332,
        0.004018384497612715,
        -0.001021946663968265,
        -0.015448919497430325,
        0.018127841874957085,
        0.009251306764781475,
        0.009154143743216991,
        0.013498718850314617,
        -0.009188844822347164,
        0.00477486802265048,
        0.017558744177222252,
        0.00261646113358438,
        0.004674234893172979,
        0.0035412448924034834,
        0.012075974605977535,
        0.009403991512954235,
        -0.02001558057963848,
        -0.0014452998293563724,
        0.0031647381838411093,
        -0.013734686188399792,
        -0.009695480577647686,
        -0.030342621728777885,
        0.011034942232072353,
        -0.011125165037810802,
        -0.0007252524956129491,
        -0.007252525072544813,
        0.0014921462861821055,
        0.025317905470728874,
        -0.03170290216803551,
        -0.007578715216368437,
        -0.02524850331246853,
        0.0029860276263207197,
        -0.010826735757291317,
        -0.011631800793111324,
        -0.02430463396012783,
        0.014178860001266003,
        0.0050837076269090176,
        -0.012894920073449612,
        0.009751002304255962,
        0.014005354605615139,
        -0.011277849785983562,
        -0.0050316560082137585,
        -0.02628953568637371,
        0.005201691295951605,
        0.00963995885103941,
        -0.012513208203017712,
        -0.01697576604783535,
        -0.008238035254180431,
        0.007203943561762571,
        -0.0012527088401839137,
        0.009237426333129406,
        -0.02552611194550991,
        0.01743382029235363,
        -0.007453791331499815,
        -0.027275046333670616,
        -0.012207838706672192,
        0.0029686770867556334,
        0.004795688670128584,
        0.00462218327447772,
        -0.001602322212420404,
        -0.017072929069399834,
        0.010438083671033382,
        0.019182754680514336,
        -0.003515219083055854,
        0.007349688094109297,
        0.0018790633184835315,
        0.036616574972867966,
        0.009556676261126995,
        0.004913672339171171,
        -0.006856932770460844,
        -0.019849015399813652,
        0.021820036694407463,
        0.05088565871119499,
        0.01772530935704708,
        -0.006936745252460241,
        -0.007314987014979124,
        0.0048789712600409985,
        -0.008085350506007671,
        -0.0060241068713366985,
        -0.011666501872241497,
        0.018113961443305016,
        -0.008827953599393368,
        -0.01911335252225399,
        -0.01866917870938778,
        -0.003733835881575942,
        -0.0036227924283593893,
        0.0031456525903195143,
        -0.008730790577828884,
        -0.0072317044250667095,
        -0.012353583239018917,
        0.004538900684565306,
        -0.012041273526847363,
        -0.01789187453687191,
        0.010916958563029766,
        0.020723482593894005,
        0.01246462669223547,
        0.014213561080396175,
        -0.014824300073087215,
        0.027941307052969933,
        0.024360155686736107,
        0.016004135832190514,
        -0.010417263023555279,
        -0.004295993130654097,
        0.010104953311383724,
        0.0018495674012228847,
        0.016184581443667412,
        0.004747107159346342,
        0.02476268820464611,
        0.017614265903830528,
        -0.00834213849157095,
        0.017933515831828117,
        -0.007550954353064299,
        -0.0017489342717453837,
        0.008258855901658535,
        0.016489950940012932,
        -0.013804088346660137,
        -0.0037234255578368902,
        -0.005895712878555059,
        -0.0020560387056320906,
        0.016587113961577415,
        0.0010800709715113044,
        -0.013713865540921688,
        0.017808591946959496,
        -0.0069645061157643795,
        -0.017655907198786736,
        -0.022583460435271263,
        0.0021185006480664015,
        0.022291971370577812,
        0.014213561080396175,
        0.006614025216549635,
        -0.02186167798936367,
        -0.05288444086909294,
        -0.017211733385920525,
        -0.019654689356684685,
        0.022999873384833336,
        0.0002550529025029391,
        -0.03178618475794792,
        0.010903078131377697,
        -0.05288444086909294,
        -0.002675452968105674,
        -0.038754161447286606,
        0.005243332590907812,
        -0.04130816087126732,
        -0.011118224821984768,
        -0.018849624320864677,
        0.010077192448079586,
        0.029898447915911674,
        0.015629364177584648,
        0.005701386835426092,
        -0.007280285935848951,
        -0.007981247268617153,
        -0.0032029093708842993,
        -0.013019843958318233,
        -0.010306219570338726,
        -0.007176182698458433,
        -0.016031896695494652,
        -0.0174477007240057,
        0.015823690220713615,
        0.002444690791890025,
        -0.007391329389065504,
        -0.0006983591592870653,
        0.0059477644972503185,
        -0.012013512663543224,
        -0.004080846440047026,
        0.012714474461972713,
        -0.004084316547960043,
        -0.008744671009480953,
        0.00979264359921217,
        0.02917666547000408,
        0.010805915109813213,
        -0.0011234473204240203,
        -0.019987819716334343,
        -0.0013238460524007678,
        -0.013047604821622372,
        -0.0015936469426378608,
        -0.004795688670128584,
        0.014796539209783077,
        -0.015671005472540855,
        0.01698964647948742,
        0.014394006691873074,
        -0.006655666511505842,
        0.0059720552526414394,
        0.015296234749257565,
        -0.01972409151494503,
        0.009376230649650097,
        0.01635114662349224,
        0.004021854605525732,
        0.017322776839137077,
        -0.005867952015250921,
        0.015337876044213772,
        -0.006044927518814802,
        0.0070026773028075695,
        -0.00721782399341464,
        -0.017544863745570183,
        0.009147203527390957,
        -0.003643613075837493,
        -0.007585655432194471,
        0.019987819716334343,
        -0.0009993909625336528,
        -0.020987210795283318,
        -0.006527272518724203,
        0.010916958563029766,
        -0.019363200291991234,
        0.022208688780665398,
        -0.0027014787774533033,
        -0.01941872201859951,
        -0.04111383482813835,
        -0.010743453167378902,
        -0.012964322231709957,
        0.013082305900752544,
        -0.012943501584231853,
        -0.013630582951009274,
        0.008883475326001644,
        -0.014394006691873074,
        0.006523802410811186,
        -0.018002917990088463,
        0.009306828491389751,
        -0.020987210795283318,
        -0.003398970467969775,
        -0.010104953311383724,
        0.008737730793654919,
        0.023138677701354027,
        0.0023891690652817488,
        0.014824300073087215,
        -0.017503222450613976,
        -0.003733835881575942,
        -0.023777177557349205,
        -0.017072929069399834,
        -0.003166473237797618,
        -0.01414415892213583,
        0.015421158634126186,
        0.029010100290179253,
        0.04483379051089287,
        -0.02048751525580883,
        0.012429925613105297,
        0.019043950363993645,
        -0.016101298853754997,
        0.007835502736270428,
        -0.024262992665171623,
        -0.005555642303079367,
        -0.008161692880094051,
        0.009737121872603893,
        0.017086809501051903,
        -0.006870813202112913,
        0.003553390270099044,
        -0.02903786115348339,
        0.02995396964251995,
        0.02658102475106716,
        -0.0009950533276423812,
        0.007897964678704739,
        -0.001700352760963142,
        -0.006908984389156103,
        -0.012964322231709957,
        0.004951843526214361,
        -0.00013262317224871367,
        0.01589309237897396,
        0.009147203527390957,
        -0.013422376476228237,
        0.024429557844996452,
        0.00311268656514585,
        0.010916958563029766,
        0.008959817700088024,
        0.013158648274838924,
        -0.0036540233995765448,
        0.0063155959360301495,
        0.007710579317063093,
        0.016018016263842583,
        -0.018558135256171227,
        0.009070861153304577,
        -0.01229806151241064,
        -0.00714148161932826,
        0.022403014823794365,
        0.0037546565290540457,
        0.016142940148711205,
        -0.007634236942976713,
        -0.006523802410811186,
        -0.0029738822486251593,
        -0.025137459859251976,
        -0.04727674648165703,
        -0.019043950363993645,
        -0.012263360433280468,
        0.006600144784897566,
        -0.01776695065200329,
        -0.010111893527209759,
        -0.01986289583146572,
        -0.0044729686342179775,
        -0.0008601529407314956,
        0.001425346708856523,
        -0.0054133678786456585,
        0.028926817700266838,
        -0.012180077843368053,
        -0.013699985109269619,
        -0.00993144791573286,
        -0.005774259101599455,
        0.03270229324698448,
        -0.0033486539032310247,
        0.005239862482994795,
        0.006232313346117735,
        0.014879821799695492,
        -0.007731399964541197,
        -0.01896066777408123,
        -0.005378666799515486,
        0.007210883777588606,
        0.030176056548953056,
        0.011416654102504253,
        -0.006915924604982138,
        -0.019960058853030205,
        0.009730181656777859,
        0.025803720578551292,
        -0.0180306788533926,
        -0.03978131338953972,
        0.011541577987372875,
        -0.005597283598035574,
        0.008293556980788708,
        -0.01944648288190365,
        -0.022083764895796776,
        -0.018447091802954674,
        0.015101908706128597,
        -0.015684885904192924,
        0.03239692375063896,
        -0.0078702038154006,
        -0.0022711853962391615,
        -0.0014479024102911353,
        0.0004940565559081733,
        -0.0345345102250576,
        0.006895103957504034,
        0.008522584103047848,
        -0.017697548493742943,
        -0.013477898202836514,
        0.02780250273644924,
        -0.02386046014726162,
        -0.01269365381449461,
        0.0030727803241461515,
        0.03658881410956383,
        -0.013429316692054272,
        -0.013790207915008068,
        -0.0008970228373073041,
        0.02019602619111538,
        -0.014477289281785488,
        -0.0014288168167695403,
        -0.017253374680876732,
        0.0281356330960989,
        -0.02950979582965374,
        -0.0039385720156133175,
        0.00855728518217802,
        0.004257821943610907,
        0.015171310864388943,
        0.011125165037810802,
        -0.01214537676423788,
        -0.015490560792386532,
        0.005073297303169966,
        0.001408863696269691,
        0.012728354893624783,
        -0.00040643636020831764,
        -0.011340311728417873,
        0.0021462615113705397,
        -0.0033156878780573606,
        -0.02950979582965374,
        0.0045354305766522884,
        -0.003942042123526335,
        0.01025069784373045,
        -0.024498960003256798,
        0.0013385940110310912,
        0.02311091683804989,
        0.019460363313555717,
        0.0013177733635529876,
        -0.0017445966368541121,
        0.003515219083055854,
        0.0011781015200540423,
        0.02494313381612301,
        0.0026771880220621824,
        0.0036158522125333548,
        0.00028238000231795013,
        -0.013526479713618755,
        0.0014956163940951228,
        0.027885785326361656,
        -0.02766369841992855,
        0.0034406117629259825,
        0.012707534246146679,
        -0.020862286910414696,
        -0.0048546805046498775,
        0.016878603026270866,
        0.0026060508098453283,
        0.008411540649831295,
        -0.0060865688137710094,
        0.0002385699044680223,
        0.030453665181994438,
        -0.0029478564392775297,
        0.02001558057963848,
        0.014546691440045834,
        -0.02097333036363125,
        0.0033035425003618,
        -0.026511622592806816,
        0.005482770036906004,
        0.020765123888850212,
        -0.010417263023555279,
        -0.002422135090455413,
        -0.011430534534156322,
        0.0005300589255057275,
        0.0032913971226662397,
        -0.023818818852305412,
        -0.001997046871110797,
        -0.005163520108908415,
        0.02399926446378231,
        -0.0005248537636362016,
        -0.02250017784535885,
        -0.0030970710795372725,
        -0.0038067081477493048,
        -0.00809923093765974,
        0.045055877417325974,
        0.023763297125697136,
        -0.0004515477630775422,
        0.0024845970328897238,
        0.043057095259428024,
        0.011284790001809597,
        0.006603614892810583,
        0.0019849014934152365,
        -0.013977593742311,
        -0.016129059717059135,
        -0.015559962950646877,
        -0.012318882159888744,
        -0.014852060936391354,
        -0.034812118858098984,
        0.02615073136985302,
        0.00269280350767076,
        -0.016476070508360863,
        -0.0288990568369627,
        -0.018252765759825706,
        0.003190763993188739,
        -0.00802288856357336,
        0.003595031565055251,
        -0.0025002125184983015,
        0.006260074209421873,
        0.007314987014979124,
        -0.0006319933454506099,
        0.0212648194283247,
        0.004247411619871855,
        0.01382490899413824,
        -0.03167514130473137,
        0.004216180648654699,
        0.0174477007240057,
        0.011055762879550457,
        0.010764273814857006,
        0.0027917015831917524,
        -0.04849822446703911,
        -0.019973939284682274,
        0.012631191872060299,
        0.02065408043563366,
        0.00046889830264262855,
        0.0218477975577116,
        0.021014971658587456,
        -0.0030259338673204184,
        -0.016462190076708794,
        -0.00013912962458562106,
        -0.014921463094651699,
        0.003051959676668048,
        0.02322196029126644,
        -0.013450137339532375,
        -0.023666134104132652,
        -0.009501154534518719,
        -0.006284364964812994,
        -0.033562880009412766,
        0.011708143167197704,
        0.018572015687823296,
        -0.0026147260796278715,
        -0.010451964102685452,
        -0.016129059717059135,
        -0.029898447915911674,
        -0.008772431872785091,
        -0.0073843891732394695,
        0.025637155398726463,
        0.02631729654967785,
        -0.025914764031767845,
        0.004990014713257551,
        0.0029582667630165815,
        -0.008668328635394573,
        -0.006600144784897566,
        -0.006763239856809378,
        -0.02215316705405712,
        0.011451355181634426,
        0.009938388131558895,
        -0.0026615725364536047,
        -0.022555699571967125,
        -0.01275611575692892,
        0.00841848086565733,
        -0.02093168906867504,
        0.0056423950009047985,
        0.018627537414431572,
        -0.024734927341341972,
        0.0030883958097547293,
        0.022583460435271263,
        0.03381272777915001,
        -0.03850431367754936,
        0.002444690791890025,
        -0.0021670821588486433,
        -0.0258453618735075,
        -0.006856932770460844,
        -0.003447551978752017,
        -0.013068425469100475,
        -0.032979901880025864,
        0.010833675973117352,
        -0.008612806908786297,
        -0.010882257483899593,
        -0.016309505328536034,
        -0.021070493385195732,
        -0.008397660218179226,
        -0.007557894568890333,
        0.002675452968105674,
        0.20676289498806,
        -0.01368610467761755,
        -0.010368681512773037,
        0.013894311152398586,
        -0.005579933058470488,
        0.006211492698639631,
        0.01458833273500204,
        0.012443806044757366,
        -0.006065748166292906,
        0.005430718418210745,
        0.0029062151443213224,
        -0.008244975470006466,
        -0.021056612953543663,
        -0.005625044461339712,
        -0.005010835360735655,
        -0.0009325914434157312,
        -0.01177754532545805,
        -0.030037252232432365,
        -0.00970242079347372,
        0.0007313251844607294,
        0.01776695065200329,
        -0.01743382029235363,
        0.012797757051885128,
        -0.0025678796228021383,
        0.011708143167197704,
        -0.007863263599574566,
        -0.0004840800247620791,
        -0.014213561080396175,
        0.010930838994681835,
        0.01635114662349224,
        -0.018460972234606743,
        -0.005350905936211348,
        -0.0007486757240258157,
        0.017031287774443626,
        0.0032011743169277906,
        -0.0021289109718054533,
        0.021514667198061943,
        -0.013602822087705135,
        0.028954578563570976,
        0.010146594606339931,
        -0.003190763993188739,
        -0.0016873398562893271,
        -0.0007196135702542961,
        -0.012860218994319439,
        -0.02796906791627407,
        0.00614903075620532,
        0.0026199312414973974,
        -0.015573843382298946,
        0.00878631230443716,
        -0.012915740720927715,
        -0.010257638059556484,
        0.012957382015883923,
        0.03869863972067833,
        0.02097333036363125,
        0.005330085288733244,
        -0.003633202752098441,
        -0.007995127700269222,
        0.00687428331002593,
        -0.0035880913492292166,
        0.011742844246327877,
        -0.001547668012790382,
        0.02219480834901333,
        -0.02981516532599926,
        0.007301106583327055,
        0.007544014137238264,
        0.015323995612561703,
        -0.009077801369130611,
        -0.00032987710437737405,
        0.020709602162241936,
        -0.017655907198786736,
        0.0025418538134545088,
        -0.009633018635213375,
        -0.017253374680876732,
        0.01683696173131466,
        0.0036609636154025793,
        -0.020612439140677452,
        0.020529156550765038,
        0.036783140152692795,
        0.0037997679319232702,
        0.03595031425356865,
        -0.012749175541102886,
        0.01682308129966259,
        0.003921221476048231,
        -0.035256292670965195,
        -0.013269691728055477,
        -0.023360764607787132,
        0.03528405353426933,
        -0.013887370936572552,
        -0.02018214575946331,
        -0.006499511655420065,
        0.0012501062592491508,
        -0.048997920006513596,
        -0.008834893815219402,
        0.022833308205008507,
        0.013346034102141857,
        0.016129059717059135,
        0.010521366260945797,
        0.006679957266896963,
        -0.032646771520376205,
        -0.04697137698531151,
        -0.01775307022035122,
        0.019946178421378136,
        -0.0034631674643605947,
        0.03361840173602104,
        -0.014255202375352383,
        -0.023166438564658165,
        -0.021625710651278496,
        0.005663215648382902,
        0.0172672551125288,
        -0.01776695065200329,
        0.0006094376440159976,
        -0.014976984821259975,
        0.013783267699182034,
        0.011555458419024944,
        0.0029166254680603743,
        0.011361132375895977,
        -0.02691415511071682,
        -0.02138974331319332,
        0.0028524284716695547,
        0.01025069784373045,
        -0.012867159210145473,
        -0.039531465619802475,
        0.001441829721443355,
        0.009355410002171993,
        0.016031896695494652,
        -0.00714148161932826,
        -0.014178860001266003,
        0.009646899066865444,
        -0.004747107159346342,
        -0.007148421835154295,
        0.010819795541465282,
        -0.03581150993704796,
        0.023749416694045067,
        -0.020848406478762627,
        -0.007419090252369642,
        -0.002904480090364814,
        -0.01116680633276701,
        -0.006992266979068518,
        -0.029537556692957878,
        0.012867159210145473,
        -0.003246285719797015,
        -5.533195144380443e-05,
        0.0032410805579274893,
        -0.0014860735973343253,
        0.017503222450613976,
        -0.034506749361753464,
        0.032036032527685165,
        0.0318417064845562,
        -0.03597807511687279,
        -0.02173675410449505,
        -0.008335198275744915,
        -0.0008796722977422178,
        -0.021320341154932976,
        -0.010361741296947002,
        0.03531181439757347,
        -0.021750634536147118,
        -0.04277948662638664,
        -0.011728963814675808,
        -0.012499327771365643,
        0.011062703095376492,
        -0.04261292144656181,
        -0.03414585813879967,
        0.00857116561383009,
        -0.004771397914737463,
        -0.02555387280881405,
        -0.0027431200724095106,
        -0.18144498765468597,
        0.00916802417486906,
        0.019529765471816063,
        -0.013790207915008068,
        0.017711428925395012,
        0.00353256962262094,
        0.026386698707938194,
        0.0082519156858325,
        -0.015559962950646877,
        -0.0027066839393228292,
        0.013221110217273235,
        -0.012055153958499432,
        -0.028191154822707176,
        -0.026400579139590263,
        0.005978995468467474,
        -0.01093777921050787,
        -0.008272736333310604,
        0.01712845079600811,
        0.02200048230588436,
        0.015421158634126186,
        0.009875926189124584,
        -0.023652253672480583,
        -0.009098622016608715,
        0.0023943742271512747,
        0.009882866404950619,
        0.009959208779036999,
        -0.004112077411264181,
        0.010153534822165966,
        -0.010833675973117352,
        -0.007516253273934126,
        -0.014255202375352383,
        -0.002049098489806056,
        0.020848406478762627,
        -0.003017258597537875,
        0.00832825805991888,
        0.013588941656053066,
        0.004934492986649275,
        -0.008529524318873882,
        -0.011749784462153912,
        -0.005902653094381094,
        0.02123705856502056,
        0.016628755256533623,
        -0.009751002304255962,
        0.022111525759100914,
        0.001578031457029283,
        0.008196393959224224,
        0.019335439428687096,
        -0.01836380921304226,
        0.009050040505826473,
        0.009528915397822857,
        0.015546082518994808,
        -0.028357720002532005,
        0.013089246116578579,
        0.0038691700901836157,
        -0.0005322277429513633,
        0.013255811296403408,
        0.012422985397279263,
        -0.0007681950810365379,
        -0.009050040505826473,
        -0.011312550865113735,
        0.0033104827161878347,
        -0.013158648274838924,
        0.01711457036435604,
        -0.0061247400008141994,
        -0.02601192705333233,
        0.0009221811196766794,
        -0.00954973604530096,
        0.007897964678704739,
        -0.001029754406772554,
        0.009195785038173199,
        0.0008900826214812696,
        0.00011787522089434788,
        0.013658343814313412,
        -0.011291730217635632,
        0.008085350506007671,
        0.014213561080396175,
        0.001027151825837791,
        0.010056371800601482,
        0.01046584453433752,
        -0.009848165325820446,
        -0.011902469210326672,
        0.03836550936102867,
        -0.016004135832190514,
        0.01299902331084013,
        0.03120320849120617,
        0.01850261352956295,
        0.018141722306609154,
        0.01590697281062603,
        -0.011590159498155117,
        -0.010049431584775448,
        0.01521295215934515,
        -0.012152316980063915,
        -0.009938388131558895,
        0.00803676899522543,
        -0.0023614082019776106,
        0.008689149282872677,
        0.000945604348089546,
        -0.007432970684021711,
        0.006398878525942564,
        -0.0025036826264113188,
        0.006051867734640837,
        0.0014053935883566737,
        -0.012152316980063915,
        0.029287708923220634,
        0.02524850331246853,
        0.014852060936391354,
        0.003261901205405593,
        0.02169511280953884,
        0.03256348893046379,
        -0.019918417558073997,
        -0.020307069644331932,
        -0.0014938813401386142,
        0.00902921985834837,
        0.021139895543456078,
        0.02399926446378231,
        0.012103735469281673,
        0.004146778490394354,
        -0.006332946475595236,
        0.022833308205008507,
        0.007474611978977919,
        0.049775224179029465,
        -0.005291914101690054,
        0.0010453698923811316,
        0.004844270180910826,
        0.027927426621317863,
        -0.026039687916636467,
        -0.12514595687389374,
        -0.012561789713799953,
        0.002030012896284461,
        0.019182754680514336,
        -0.0028437532018870115,
        0.002788231475278735,
        0.011083523742854595,
        0.02630341611802578,
        -0.018447091802954674,
        0.015823690220713615,
        0.002512357896193862,
        0.001597117050550878,
        -0.00948727410286665,
        0.0007625561556778848,
        0.008293556980788708,
        0.020612439140677452,
        0.006447460036724806,
        -0.006326006259769201,
        -0.014768778346478939,
        0.019696330651640892,
        0.0034544921945780516,
        0.017100689932703972,
        0.021209297701716423,
        0.003333038417622447,
        -0.00653074262663722,
        0.004504199605435133,
        -0.02767757885158062,
        0.014352365396916866,
        0.010091072879731655,
        -0.010306219570338726,
        0.02308315597474575,
        -0.009862045757472515,
        -0.010889197699725628,
        -0.012339702807366848,
        -0.002030012896284461,
        -0.01154851820319891,
        -0.029398752376437187,
        -0.005781199317425489,
        0.010410322807729244,
        -0.013769387267529964,
        -0.0008007273427210748,
        -0.00397327309474349,
        0.024804329499602318,
        0.015268473885953426,
        0.0210843738168478,
        -0.01460221316665411,
        -0.015809809789061546,
        -0.002512357896193862,
        0.007176182698458433,
        -0.0227916669100523,
        -0.02383269928395748,
        -0.018474852666258812,
        -0.023777177557349205,
        -0.021820036694407463,
        0.02111213468015194,
        0.012318882159888744,
        0.010000850073993206,
        0.009528915397822857,
        -0.018127841874957085,
        0.005951234605163336,
        -0.00031751484493725,
        -0.012013512663543224,
        -0.015351756475865841,
        0.014227441512048244,
        0.005340495612472296,
        0.007398269604891539,
        -0.03445122763514519,
        -0.027941307052969933,
        0.009063920937478542,
        -0.007356628309935331,
        -0.0003285758139099926,
        0.03747716173529625,
        -0.017086809501051903,
        0.017794711515307426,
        -0.011048822663724422,
        -0.001525979838334024,
        -0.012978202663362026,
        -0.0070026773028075695,
        -0.0024672464933246374,
        -0.0020751242991536856,
        -0.011506876908242702,
        -0.017017407342791557,
        -0.001215405180118978,
        -0.01061852928251028,
        0.01944648288190365,
        0.021792275831103325,
        0.02784414403140545,
        -0.006409288849681616,
        -0.017614265903830528,
        -0.015532202087342739,
        0.003556860378012061,
        0.003695664694532752,
        -0.025317905470728874,
        -0.013505659066140652,
        -0.04377887770533562,
        0.010611589066684246,
        -0.015310115180909634,
        -0.01712845079600811,
        0.008598926477134228,
        -0.005323145072907209,
        -0.013637523166835308,
        -0.01376244705170393,
        -0.013977593742311,
        0.022278090938925743,
        -0.02077900432050228,
        -0.009452573023736477,
        -0.0007525795954279602,
        -0.018752461299300194,
        -0.003529099514707923,
        -0.0005955572123639286,
        -0.037699248641729355,
        0.007405209820717573,
        -0.0007642912096343935,
        0.004115547519177198,
        -0.033285271376371384,
        -0.02447119913995266,
        -0.010528306476771832,
        0.00515657989308238,
        0.01161098014563322,
        -0.003962862771004438,
        0.01836380921304226,
        0.014768778346478939,
        -0.0210843738168478,
        -0.007009617518633604,
        0.02323584072291851,
        0.01790575496852398,
        -0.024804329499602318,
        0.0022937410976737738,
        -0.03717179223895073,
        0.027025198563933372,
        -0.0020699191372841597,
        0.0005864481790922582,
        0.006725068669766188,
        -0.02783026359975338,
        -0.005659745540469885,
        0.003900401061400771,
        -0.0010878787143155932,
        -0.009695480577647686,
        0.015684885904192924,
        0.0033885601442307234,
        0.014616093598306179,
        0.007224764209240675,
        -0.03620016202330589,
        -0.015351756475865841,
        -0.005618104245513678,
        -9.152408892987296e-05,
        -0.0288990568369627,
        0.004580541979521513,
        0.0024013144429773092,
        0.0040149143896996975,
        0.017530983313918114,
        -0.003921221476048231,
        0.035422857850790024,
        0.013241930864751339,
        0.032036032527685165,
        -0.041807856410741806,
        -0.010278458707034588,
        0.0037026049103587866,
        0.0029443863313645124,
        0.00180445599835366,
        0.0031456525903195143,
        -0.01605965755879879,
        0.016267864033579826,
        0.006069218274205923,
        0.029759643599390984,
        -0.011041882447898388,
        -0.01941872201859951,
        0.011812246404588223,
        0.01636502705514431,
        -0.012894920073449612,
        -0.004771397914737463,
        -0.03625568374991417,
        -0.0067354789935052395,
        0.0022243389394134283,
        0.012180077843368053,
        0.010118833743035793,
        0.015976374968886375,
        0.0026338116731494665,
        0.0009221811196766794,
        0.005264153238385916,
        -0.014616093598306179,
        0.015504441224038601,
        0.014407887123525143,
        0.0022659802343696356,
        -0.016795320436358452,
        -0.0018964138580486178,
        0.02705295942723751,
        0.010333980433642864,
        0.004715876188129187,
        0.004039205145090818,
        0.004108607303351164,
        0.014643854461610317,
        -0.0210843738168478,
        -0.0011217122664675117,
        0.008120051585137844,
        0.005645865108817816,
        0.007391329389065504,
        0.02827443741261959,
        -0.019321558997035027,
        0.012783876620233059,
        -0.009730181656777859,
        0.016462190076708794,
        -0.003461432410404086,
        0.006954095792025328,
        -0.012138436548411846,
        -0.003079720539972186,
        -0.006579324137419462,
        -0.007703639101237059,
        -0.014727137051522732,
        0.0025036826264113188,
        0.007793861906975508,
        0.012499327771365643,
        0.0036262625362724066,
        0.006718128453940153,
        0.017197852954268456,
        0.01865529827773571,
        -0.03869863972067833,
        0.022985992953181267,
        -0.0010748658096417785,
        -0.012166197411715984,
        -0.010354801081120968,
        0.036644335836172104,
        0.025873122736811638,
        0.00857116561383009,
        0.012610371224582195,
        0.010882257483899593,
        0.03667209669947624,
        0.006346826907247305,
        0.0015112318797037005,
        -0.010951659642159939,
        0.029121143743395805,
        0.008578105829656124,
        0.0029634719248861074,
        -0.016476070508360863,
        -0.007939605973660946,
        -0.017350537702441216,
        -0.021667351946234703,
        -0.02735832892358303,
        -0.0026615725364536047,
        0.05094118043780327,
        -0.009744062088429928,
        0.06013002619147301,
        0.0009881131118163466,
        -0.021292580291628838,
        0.012110675685107708,
        -0.008751611225306988,
        0.02783026359975338,
        -0.0057187373749911785,
        0.00918190460652113,
        -0.004178009461611509,
        -0.008911236189305782,
        0.011145985685288906,
        -0.016781440004706383,
        0.003969802986830473,
        -0.04466722533106804,
        -0.013089246116578579,
        -0.024651644751429558,
        -0.0008415011106990278,
        0.004823449533432722,
        -0.016295624896883965,
        -0.008487883023917675,
        0.03350735828280449,
        -0.009653839282691479,
        0.017059048637747765,
        0.0061004492454230785,
        -0.007467671763151884,
        -0.014095577411353588,
        0.013970653526484966,
        -0.04286276921629906,
        -0.018252765759825706,
        -0.0339515320956707,
        -0.0014652529498562217,
        -0.008550344966351986,
        -0.02906562201678753,
        -0.019835134968161583,
        -0.012117615900933743,
        0.030342621728777885,
        0.004094726871699095,
        -0.01191634964197874,
        0.01604577712714672,
        0.013588941656053066,
        -0.008050649426877499,
        0.008654448203742504,
        -0.01222171913832426,
        -0.02752489410340786,
        0.0038032380398362875,
        -0.018752461299300194,
        -0.01760038547217846,
        0.00030450194026343524,
        -0.006398878525942564
      ],
      "index": 1,
      "object": "embedding"
    }
  ],
  "model": "text-embedding-ada-002-v2",
  "object": "list",
  "usage": {
    "prompt_tokens": 13,
    "total_tokens": 13
  }
}
print(f"vector 0: {len(res['data'][0]['embedding'])}\nvector 1: {len(res['data'][1]['embedding'])}")
vector 0: 1536
vector 1: 1536

# we can extract embeddings to a list
embeds = [record['embedding'] for record in res['data']]
len(embeds)
2

Next, we initialize our index to store vector embeddings with Pinecone.

len(embeds[0])
1536
import pinecone

index_name = 'semantic-search-openai'

# initialize connection to pinecone (get API key at app.pinecone.io)
pinecone.init(
    api_key="PINECONE_API_KEY",
    environment="PINECONE_ENVIRONMENT"  # find next to api key in console
)
# check if 'openai' index already exists (only create index if not)
if index_name not in pinecone.list_indexes():
    pinecone.create_index(index_name, dimension=len(embeds[0]))
# connect to index
index = pinecone.Index(index_name)
from datasets import load_dataset

# load the first 1K rows of the TREC dataset
trec = load_dataset('trec', split='train[:1000]')
trec
WARNING:datasets.builder:Found cached dataset trec (/root/.cache/huggingface/datasets/trec/default/2.0.0/f2469cab1b5fceec7249fda55360dfdbd92a7a5b545e91ea0f78ad108ffac1c2)

Dataset({
    features: ['text', 'coarse_label', 'fine_label'],
    num_rows: 1000
})
trec[0]
{'text': 'How did serfdom develop in and then leave Russia ?',
 'coarse_label': 2,
 'fine_label': 26}

Then we create a vector embedding for each phrase using OpenAI, and upsert the ID, vector embedding, and original text for each phrase to Pinecone.

from tqdm.auto import tqdm

count = 0  # we'll use the count to create unique IDs
batch_size = 32  # process everything in batches of 32
for i in tqdm(range(0, len(trec['text']), batch_size)):
    # set end position of batch
    i_end = min(i+batch_size, len(trec['text']))
    # get batch of lines and IDs
    lines_batch = trec['text'][i: i+batch_size]
    ids_batch = [str(n) for n in range(i, i_end)]
    # create embeddings
    res = openai.Embedding.create(input=lines_batch, engine=MODEL)
    embeds = [record['embedding'] for record in res['data']]
    # prep metadata and upsert batch
    meta = [{'text': line} for line in lines_batch]
    to_upsert = zip(ids_batch, embeds, meta)
    # upsert to Pinecone
    index.upsert(vectors=list(to_upsert))
  0%|          | 0/32 [00:00<?, ?it/s]

Querying

With our data indexed, we're now ready to move onto performing searches. This follows a similar process to indexing. We start with a text query, that we would like to use to find similar sentences. As before we encode this with OpenAI's text similarity Babbage model to create a query vector xq. We then use xq to query the Pinecone index.

query = "What caused the 1929 Great Depression?"

xq = openai.Embedding.create(input=query, engine=MODEL)['data'][0]['embedding']

Now query...

res = index.query([xq], top_k=5, include_metadata=True)
res
{'matches': [{'id': '932',
              'metadata': {'text': 'Why did the world enter a global '
                                   'depression in 1929 ?'},
              'score': 0.917971551,
              'sparseValues': {},
              'values': []},
             {'id': '787',
              'metadata': {'text': "When was `` the Great Depression '' ?"},
              'score': 0.87167418,
              'sparseValues': {},
              'values': []},
             {'id': '400',
              'metadata': {'text': 'What crop failure caused the Irish Famine '
                                   '?'},
              'score': 0.812044263,
              'sparseValues': {},
              'values': []},
             {'id': '775',
              'metadata': {'text': 'What historical event happened in Dogtown '
                                   'in 1899 ?'},
              'score': 0.798895657,
              'sparseValues': {},
              'values': []},
             {'id': '481',
              'metadata': {'text': 'What caused the Lynmouth floods ?'},
              'score': 0.792350292,
              'sparseValues': {},
              'values': []}],
 'namespace': ''}

The response from Pinecone includes our original text in the metadata field, let's print out the top_k most similar questions and their respective similarity scores.

for match in res['matches']:
    print(f"{match['score']:.2f}: {match['metadata']['text']}")
0.92: Why did the world enter a global depression in 1929 ?
0.87: When was `` the Great Depression '' ?
0.81: What crop failure caused the Irish Famine ?
0.80: What historical event happened in Dogtown in 1899 ?
0.79: What caused the Lynmouth floods ?

Looks good, let's make it harder and replace "depression" with the incorrect term "recession".

query = "What was the cause of the major recession in the early 20th century?"

# create the query embedding
xq = openai.Embedding.create(input=query, engine=MODEL)['data'][0]['embedding']

# query, returning the top 5 most similar results
res = index.query([xq], top_k=5, include_metadata=True)

for match in res['matches']:
    print(f"{match['score']:.2f}: {match['metadata']['text']}")
0.88: Why did the world enter a global depression in 1929 ?
0.83: When was `` the Great Depression '' ?
0.81: What crop failure caused the Irish Famine ?
0.80: When did World War I start ?
0.80: What were popular songs and types of songs in the 1920s ?

And again...

query = "Why was there a long-term economic downturn in the early 20th century?"

# create the query embedding
xq = openai.Embedding.create(input=query, engine=MODEL)['data'][0]['embedding']

# query, returning the top 5 most similar results
res = index.query([xq], top_k=5, include_metadata=True)

for match in res['matches']:
    print(f"{match['score']:.2f}: {match['metadata']['text']}")
0.90: Why did the world enter a global depression in 1929 ?
0.84: When was `` the Great Depression '' ?
0.80: When did World War I start ?
0.80: What crop failure caused the Irish Famine ?
0.80: When did the Dow first reach ?

Looks great, our semantic search pipeline is clearly able to identify the meaning between each of our queries and return the most semantically similar questions from the already indexed questions.

Once we're finished with the index we delete it to save resources.

pinecone.delete_index(index_name)