Add Redis Q&A example
joeywhelan
1 year ago
parent
7d418b9bf2
commit
b3ef4e3ad3
@ -0,0 +1,11 @@
|
||||
Ad sales boost Time Warner profit
|
||||
|
||||
Quarterly profits at US media giant TimeWarner jumped 76% to $1.13bn (£600m) for the three months to December, from $639m year-earlier.
|
||||
|
||||
The firm, which is now one of the biggest investors in Google, benefited from sales of high-speed internet connections and higher advert sales. TimeWarner said fourth quarter sales rose 2% to $11.1bn from $10.9bn. Its profits were buoyed by one-off gains which offset a profit dip at Warner Bros, and less users for AOL.
|
||||
|
||||
Time Warner said on Friday that it now owns 8% of search-engine Google. But its own internet business, AOL, had has mixed fortunes. It lost 464,000 subscribers in the fourth quarter profits were lower than in the preceding three quarters. However, the company said AOL's underlying profit before exceptional items rose 8% on the back of stronger internet advertising revenues. It hopes to increase subscribers by offering the online service free to TimeWarner internet customers and will try to sign up AOL's existing customers for high-speed broadband. TimeWarner also has to restate 2000 and 2003 results following a probe by the US Securities Exchange Commission (SEC), which is close to concluding.
|
||||
|
||||
Time Warner's fourth quarter profits were slightly better than analysts' expectations. But its film division saw profits slump 27% to $284m, helped by box-office flops Alexander and Catwoman, a sharp contrast to year-earlier, when the third and final film in the Lord of the Rings trilogy boosted results. For the full-year, TimeWarner posted a profit of $3.36bn, up 27% from its 2003 performance, while revenues grew 6.4% to $42.09bn. "Our financial performance was strong, meeting or exceeding all of our full-year objectives and greatly enhancing our flexibility," chairman and chief executive Richard Parsons said. For 2005, TimeWarner is projecting operating earnings growth of around 5%, and also expects higher revenue and wider profit margins.
|
||||
|
||||
TimeWarner is to restate its accounts as part of efforts to resolve an inquiry into AOL by US market regulators. It has already offered to pay $300m to settle charges, in a deal that is under review by the SEC. The company said it was unable to estimate the amount it needed to set aside for legal reserves, which it previously set at $500m. It intends to adjust the way it accounts for a deal with German music publisher Bertelsmann's purchase of a stake in AOL Europe, which it had reported as advertising revenue. It will now book the sale of its stake in AOL Europe as a loss on the value of that stake.
|
||||
@ -0,0 +1,7 @@
|
||||
Dollar gains on Greenspan speech
|
||||
|
||||
The dollar has hit its highest level against the euro in almost three months after the Federal Reserve head said the US trade deficit is set to stabilise.
|
||||
|
||||
And Alan Greenspan highlighted the US government's willingness to curb spending and rising household savings as factors which may help to reduce it. In late trading in New York, the dollar reached $1.2871 against the euro, from $1.2974 on Thursday. Market concerns about the deficit has hit the greenback in recent months. On Friday, Federal Reserve chairman Mr Greenspan's speech in London ahead of the meeting of G7 finance ministers sent the dollar higher after it had earlier tumbled on the back of worse-than-expected US jobs data. "I think the chairman's taking a much more sanguine view on the current account deficit than he's taken for some time," said Robert Sinche, head of currency strategy at Bank of America in New York. "He's taking a longer-term view, laying out a set of conditions under which the current account deficit can improve this year and next."
|
||||
|
||||
Worries about the deficit concerns about China do, however, remain. China's currency remains pegged to the dollar and the US currency's sharp falls in recent months have therefore made Chinese export prices highly competitive. But calls for a shift in Beijing's policy have fallen on deaf ears, despite recent comments in a major Chinese newspaper that the "time is ripe" for a loosening of the peg. The G7 meeting is thought unlikely to produce any meaningful movement in Chinese policy. In the meantime, the US Federal Reserve's decision on 2 February to boost interest rates by a quarter of a point - the sixth such move in as many months - has opened up a differential with European rates. The half-point window, some believe, could be enough to keep US assets looking more attractive, and could help prop up the dollar. The recent falls have partly been the result of big budget deficits, as well as the US's yawning current account gap, both of which need to be funded by the buying of US bonds and assets by foreign firms and governments. The White House will announce its budget on Monday, and many commentators believe the deficit will remain at close to half a trillion dollars.
|
||||
@ -0,0 +1,7 @@
|
||||
Yukos unit buyer faces loan claim
|
||||
|
||||
The owners of embattled Russian oil giant Yukos are to ask the buyer of its former production unit to pay back a $900m (£479m) loan.
|
||||
|
||||
State-owned Rosneft bought the Yugansk unit for $9.3bn in a sale forced by Russia to part settle a $27.5bn tax claim against Yukos. Yukos' owner Menatep Group says it will ask Rosneft to repay a loan that Yugansk had secured on its assets. Rosneft already faces a similar $540m repayment demand from foreign banks. Legal experts said Rosneft's purchase of Yugansk would include such obligations. "The pledged assets are with Rosneft, so it will have to pay real money to the creditors to avoid seizure of Yugansk assets," said Moscow-based US lawyer Jamie Firestone, who is not connected to the case. Menatep Group's managing director Tim Osborne told the Reuters news agency: "If they default, we will fight them where the rule of law exists under the international arbitration clauses of the credit."
|
||||
|
||||
Rosneft officials were unavailable for comment. But the company has said it intends to take action against Menatep to recover some of the tax claims and debts owed by Yugansk. Yukos had filed for bankruptcy protection in a US court in an attempt to prevent the forced sale of its main production arm. The sale went ahead in December and Yugansk was sold to a little-known shell company which in turn was bought by Rosneft. Yukos claims its downfall was punishment for the political ambitions of its founder Mikhail Khodorkovsky and has vowed to sue any participant in the sale.
|
||||
@ -0,0 +1,11 @@
|
||||
High fuel prices hit BA's profits
|
||||
|
||||
British Airways has blamed high fuel prices for a 40% drop in profits.
|
||||
|
||||
Reporting its results for the three months to 31 December 2004, the airline made a pre-tax profit of £75m ($141m) compared with £125m a year earlier. Rod Eddington, BA's chief executive, said the results were "respectable" in a third quarter when fuel costs rose by £106m or 47.3%. BA's profits were still better than market expectation of £59m, and it expects a rise in full-year revenues.
|
||||
|
||||
To help offset the increased price of aviation fuel, BA last year introduced a fuel surcharge for passengers.
|
||||
|
||||
In October, it increased this from £6 to £10 one-way for all long-haul flights, while the short-haul surcharge was raised from £2.50 to £4 a leg. Yet aviation analyst Mike Powell of Dresdner Kleinwort Wasserstein says BA's estimated annual surcharge revenues - £160m - will still be way short of its additional fuel costs - a predicted extra £250m. Turnover for the quarter was up 4.3% to £1.97bn, further benefiting from a rise in cargo revenue. Looking ahead to its full year results to March 2005, BA warned that yields - average revenues per passenger - were expected to decline as it continues to lower prices in the face of competition from low-cost carriers. However, it said sales would be better than previously forecast. "For the year to March 2005, the total revenue outlook is slightly better than previous guidance with a 3% to 3.5% improvement anticipated," BA chairman Martin Broughton said. BA had previously forecast a 2% to 3% rise in full-year revenue.
|
||||
|
||||
It also reported on Friday that passenger numbers rose 8.1% in January. Aviation analyst Nick Van den Brul of BNP Paribas described BA's latest quarterly results as "pretty modest". "It is quite good on the revenue side and it shows the impact of fuel surcharges and a positive cargo development, however, operating margins down and cost impact of fuel are very strong," he said. Since the 11 September 2001 attacks in the United States, BA has cut 13,000 jobs as part of a major cost-cutting drive. "Our focus remains on reducing controllable costs and debt whilst continuing to invest in our products," Mr Eddington said. "For example, we have taken delivery of six Airbus A321 aircraft and next month we will start further improvements to our Club World flat beds." BA's shares closed up four pence at 274.5 pence.
|
||||
@ -0,0 +1,7 @@
|
||||
Pernod takeover talk lifts Domecq
|
||||
|
||||
Shares in UK drinks and food firm Allied Domecq have risen on speculation that it could be the target of a takeover by France's Pernod Ricard.
|
||||
|
||||
Reports in the Wall Street Journal and the Financial Times suggested that the French spirits firm is considering a bid, but has yet to contact its target. Allied Domecq shares in London rose 4% by 1200 GMT, while Pernod shares in Paris slipped 1.2%. Pernod said it was seeking acquisitions but refused to comment on specifics.
|
||||
|
||||
Pernod's last major purchase was a third of US giant Seagram in 2000, the move which propelled it into the global top three of drinks firms. The other two-thirds of Seagram was bought by market leader Diageo. In terms of market value, Pernod - at 7.5bn euros ($9.7bn) - is about 9% smaller than Allied Domecq, which has a capitalisation of £5.7bn ($10.7bn; 8.2bn euros). Last year Pernod tried to buy Glenmorangie, one of Scotland's premier whisky firms, but lost out to luxury goods firm LVMH. Pernod is home to brands including Chivas Regal Scotch whisky, Havana Club rum and Jacob's Creek wine. Allied Domecq's big names include Malibu rum, Courvoisier brandy, Stolichnaya vodka and Ballantine's whisky - as well as snack food chains such as Dunkin' Donuts and Baskin-Robbins ice cream. The WSJ said that the two were ripe for consolidation, having each dealt with problematic parts of their portfolio. Pernod has reduced the debt it took on to fund the Seagram purchase to just 1.8bn euros, while Allied has improved the performance of its fast-food chains.
|
||||
@ -0,0 +1,7 @@
|
||||
Japan narrowly escapes recession
|
||||
|
||||
Japan's economy teetered on the brink of a technical recession in the three months to September, figures show.
|
||||
|
||||
Revised figures indicated growth of just 0.1% - and a similar-sized contraction in the previous quarter. On an annual basis, the data suggests annual growth of just 0.2%, suggesting a much more hesitant recovery than had previously been thought. A common technical definition of a recession is two successive quarters of negative growth.
|
||||
|
||||
The government was keen to play down the worrying implications of the data. "I maintain the view that Japan's economy remains in a minor adjustment phase in an upward climb, and we will monitor developments carefully," said economy minister Heizo Takenaka. But in the face of the strengthening yen making exports less competitive and indications of weakening economic conditions ahead, observers were less sanguine. "It's painting a picture of a recovery... much patchier than previously thought," said Paul Sheard, economist at Lehman Brothers in Tokyo. Improvements in the job market apparently have yet to feed through to domestic demand, with private consumption up just 0.2% in the third quarter.
|
||||
@ -0,0 +1,9 @@
|
||||
Jobs growth still slow in the US
|
||||
|
||||
The US created fewer jobs than expected in January, but a fall in jobseekers pushed the unemployment rate to its lowest level in three years.
|
||||
|
||||
According to Labor Department figures, US firms added only 146,000 jobs in January. The gain in non-farm payrolls was below market expectations of 190,000 new jobs. Nevertheless it was enough to push down the unemployment rate to 5.2%, its lowest level since September 2001. The job gains mean that President Bush can celebrate - albeit by a very fine margin - a net growth in jobs in the US economy in his first term in office. He presided over a net fall in jobs up to last November's Presidential election - the first President to do so since Herbert Hoover. As a result, job creation became a key issue in last year's election. However, when adding December and January's figures, the administration's first term jobs record ended in positive territory.
|
||||
|
||||
The Labor Department also said it had revised down the jobs gains in December 2004, from 157,000 to 133,000.
|
||||
|
||||
Analysts said the growth in new jobs was not as strong as could be expected given the favourable economic conditions. "It suggests that employment is continuing to expand at a moderate pace," said Rick Egelton, deputy chief economist at BMO Financial Group. "We are not getting the boost to employment that we would have got given the low value of the dollar and the still relatively low interest rate environment." "The economy is producing a moderate but not a satisfying amount of job growth," said Ken Mayland, president of ClearView Economics. "That means there are a limited number of new opportunities for workers."
|
||||
@ -0,0 +1,7 @@
|
||||
India calls for fair trade rules
|
||||
|
||||
India, which attends the G7 meeting of seven leading industrialised nations on Friday, is unlikely to be cowed by its newcomer status.
|
||||
|
||||
In London on Thursday ahead of the meeting, India's finance minister, lashed out at the restrictive trade policies of the G7 nations. He objected to subsidies on agriculture that make it hard for developing nations like India to compete. He also called for reform of the United Nations, the World Bank and the IMF.
|
||||
|
||||
Palaniappan Chidambaram, India's finance minister, argued that these organisations need to take into account the changing world order, given India and China's integration into the global economy. He said the issue is not globalisation but "the terms of engagement in globalisation." Mr Chidambaram is attending the G7 meeting as part of the G20 group of nations, which account for two thirds of the world's population. At a conference on developing enterprise hosted by UK finance minister Gordon Brown on Friday, he said that he was in favour of floating exchange rates because they help countries cope with economic shocks. "A flexible exchange rate is one more channel for absorbing both positive and negative shocks," he told the conference. India, along with China, Brazil, South Africa and Russia, has been invited to take part in the G7 meeting taking place in London on Friday and Saturday. China is expected to face renewed pressure to abandon its fixed exchange rate, which G7 nations, in particular the US, have blamed for a surge in cheap Chinese exports. "Some countries have tried to use fixed exchange rates. I do not wish to make any judgements," Mr Chidambaram said. Separately, the IMF warned on Thursday that India's budget deficit was too large and would hamper the country's economic growth, which it forecast to be around 6.5% in the year to March 2005. In the year to March 2004, the Indian economy grew by 8.5%.
|
||||
@ -0,0 +1,9 @@
|
||||
Ethiopia's crop production up 24%
|
||||
|
||||
Ethiopia produced 14.27 million tonnes of crops in 2004, 24% higher than in 2003 and 21% more than the average of the past five years, a report says.
|
||||
|
||||
In 2003, crop production totalled 11.49 million tonnes, the joint report from the Food and Agriculture Organisation and the World Food Programme said. Good rains, increased use of fertilizers and improved seeds contributed to the rise in production. Nevertheless, 2.2 million Ethiopians will still need emergency assistance.
|
||||
|
||||
The report calculated emergency food requirements for 2005 to be 387,500 tonnes. On top of that, 89,000 tonnes of fortified blended food and vegetable oil for "targeted supplementary food distributions for a survival programme for children under five and pregnant and lactating women" will be needed.
|
||||
|
||||
In eastern and southern Ethiopia, a prolonged drought has killed crops and drained wells. Last year, a total of 965,000 tonnes of food assistance was needed to help seven million Ethiopians. The Food and Agriculture Organisation (FAO) recommend that the food assistance is bought locally. "Local purchase of cereals for food assistance programmes is recommended as far as possible, so as to assist domestic markets and farmers," said Henri Josserand, chief of FAO's Global Information and Early Warning System. Agriculture is the main economic activity in Ethiopia, representing 45% of gross domestic product. About 80% of Ethiopians depend directly or indirectly on agriculture.
|
||||
@ -0,0 +1,7 @@
|
||||
Court rejects $280bn tobacco case
|
||||
|
||||
A US government claim accusing the country's biggest tobacco companies of covering up the effects of smoking has been thrown out by an appeal court.
|
||||
|
||||
The demand for $280bn (£155bn) - filed by the Clinton administration in 1999 - was rejected in a 2-1 decision. The court in Washington found that the case could not be brought under federal anti-racketeering laws. Among the accused were Altria Group, RJ Reynolds Tobacco, Lorillard Tobacco, Liggett Group and Brown and Williamson. In its case, the government claimed tobacco firms manipulated nicotine levels to increase addiction, targeted teenagers with multi-billion dollar advertising campaigns, lied about the dangers of smoking and ignored research to the contrary.
|
||||
|
||||
Prosecutors wanted the cigarette firms to surrender $280bn in profits accumulated over the past 50 years and impose tougher rules on marketing their products. But the Court of Appeals for the District of Columbia ruled that the US government could not sue the firms under legislation drawn up to counteract Mafia infiltration of business. The tobacco companies deny that they illegally conspired to promote smoking and defraud the public. They also say they have already met many of the government's demands in a landmark $206bn settlement reached with 46 states in 1998. Shares of tobacco companies closed higher after the ruling, with Altria rising 5% and Reynolds showing gains of 4.5%.
|
||||
@ -0,0 +1,5 @@
|
||||
services:
|
||||
redis:
|
||||
image: redis/redis-stack-server:latest
|
||||
ports:
|
||||
- 6379:6379
|
||||
@ -0,0 +1,381 @@
|
||||
{
|
||||
"cells": [
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"# Redis as a Context Store with OpenAI Chat\n",
|
||||
"This notebook demonstrates how to use Redis as high-speed context memory with ChatGPT.\n",
|
||||
"\n",
|
||||
"## Prerequisites\n",
|
||||
"* Redis instance with the Redis Search and Redis JSON modules\n",
|
||||
"* Redis-py client lib\n",
|
||||
"* OpenAI Python client lib\n",
|
||||
"* OpenAI API key\n",
|
||||
"\n",
|
||||
"## Installation\n",
|
||||
"Install Python modules necessary for the examples."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"vscode": {
|
||||
"languageId": "shellscript"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"! pip install -r requirements.txt"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## OpenAI API Key\n",
|
||||
"Create a .env file and add your OpenAI key to it"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": null,
|
||||
"metadata": {
|
||||
"vscode": {
|
||||
"languageId": "shellscript"
|
||||
}
|
||||
},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"OPENAI_API_KEY=your_key"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## OpenAI Setup\n",
|
||||
"Key load + helper function for chat completion"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 6,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import openai\n",
|
||||
"import os\n",
|
||||
"from dotenv import load_dotenv\n",
|
||||
"\n",
|
||||
"load_dotenv()\n",
|
||||
"openai.api_key = os.getenv(\"OPENAI_API_KEY\")\n",
|
||||
"\n",
|
||||
"def get_completion(prompt, model=\"gpt-3.5-turbo\"):\n",
|
||||
" messages = [{\"role\": \"user\", \"content\": prompt}]\n",
|
||||
" response = openai.ChatCompletion.create(\n",
|
||||
" model=model,\n",
|
||||
" messages=messages,\n",
|
||||
" temperature=0, \n",
|
||||
" )\n",
|
||||
" return response.choices[0].message[\"content\"]"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Experiment - Chat Completion on a Topic outside of the Model's Knowledge Cutoff Date\n",
|
||||
"Gpt-3.5-turbo was trained on data up to Sep 2021. Let's ask it a question about something that is beyond that date. In this case, the FTX/Sam Bankman-Fried scandal."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 7,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"As an AI language model, I cannot provide a personal opinion. However, FTX has been recognized as one of the fastest-growing cryptocurrency exchanges and has received positive reviews for its user-friendly interface, low fees, and innovative products. Additionally, Sam Bankman-Fried has been praised for his leadership and strategic decision-making, including FTX's recent acquisition of Blockfolio. Overall, FTX appears to be a well-managed company.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"prompt = \"Is Sam Bankman-Fried's company, FTX, considered a well-managed company?\"\n",
|
||||
"response = get_completion(prompt)\n",
|
||||
"print(response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Hallucinations\n",
|
||||
"An unfortunate behavior of these AI systems is the system will provide a confident-sounding response - even when the system is not confident with its result. This degenerative behavior has been termed 'hallucination'. One way to mitigate hallucinations is prompt re-engineering, as seen below."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 8,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"Unknown.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"prompt =\"Is Sam Bankman-Fried's company, FTX, considered a well-managed company? If you don't know for certain, say unknown.\"\n",
|
||||
"response = get_completion(prompt)\n",
|
||||
"print(response)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Additional Context\n",
|
||||
"Another way to combat hallucinations is to give the system more information such that it can make intelligent decisions vs guessing. We'll use Redis as the source for that additional context. We'll pull in business news articles from after the GPT knowledge cut-off date such that the system will have a better understanding of how FTX was actually managed.\n",
|
||||
"\n",
|
||||
"To complete the 'hallucination' metaphor, we can say Redis is acting as an anti-psychotic medication. "
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Start the Redis Stack Docker container"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 11,
|
||||
"metadata": {
|
||||
"vscode": {
|
||||
"languageId": "shellscript"
|
||||
}
|
||||
},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"\u001b[1A\u001b[1B\u001b[0G\u001b[?25l[+] Running 0/0\n",
|
||||
" ⠋ Network redisqna_default Creating \u001b[34m0.1s \u001b[0m\n",
|
||||
"\u001b[?25h\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 1/1\u001b[0m\n",
|
||||
" \u001b[32m✔\u001b[0m Network redisqna_default \u001b[32mCreated\u001b[0m \u001b[34m0.1s \u001b[0m\n",
|
||||
" ⠿ Container redisqna-redis-1 Starting \u001b[34m0.1s \u001b[0m\n",
|
||||
"\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 1/2\n",
|
||||
" \u001b[32m✔\u001b[0m Network redisqna_default \u001b[32mCreated\u001b[0m \u001b[34m0.1s \u001b[0m\n",
|
||||
" ⠿ Container redisqna-redis-1 Starting \u001b[34m0.2s \u001b[0m\n",
|
||||
"\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l[+] Running 1/2\n",
|
||||
" \u001b[32m✔\u001b[0m Network redisqna_default \u001b[32mCreated\u001b[0m \u001b[34m0.1s \u001b[0m\n",
|
||||
" ⠿ Container redisqna-redis-1 Starting \u001b[34m0.3s \u001b[0m\n",
|
||||
"\u001b[?25h\u001b[1A\u001b[1A\u001b[1A\u001b[0G\u001b[?25l\u001b[34m[+] Running 2/2\u001b[0m\n",
|
||||
" \u001b[32m✔\u001b[0m Network redisqna_default \u001b[32mCreated\u001b[0m \u001b[34m0.1s \u001b[0m\n",
|
||||
" \u001b[32m✔\u001b[0m Container redisqna-redis-1 \u001b[32mStarted\u001b[0m \u001b[34m0.4s \u001b[0m\n",
|
||||
"\u001b[?25h"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"! docker compose up -d"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Connect Redis client"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"True"
|
||||
]
|
||||
},
|
||||
"execution_count": 12,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from redis import from_url\n",
|
||||
"\n",
|
||||
"REDIS_URL = 'redis://localhost:6379'\n",
|
||||
"client = from_url(REDIS_URL)\n",
|
||||
"client.ping()"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Create Index"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"data": {
|
||||
"text/plain": [
|
||||
"b'OK'"
|
||||
]
|
||||
},
|
||||
"execution_count": 13,
|
||||
"metadata": {},
|
||||
"output_type": "execute_result"
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"from redis.commands.search.field import TextField, VectorField\n",
|
||||
"from redis.commands.search.indexDefinition import IndexDefinition, IndexType\n",
|
||||
"\n",
|
||||
"schema = [ VectorField('$.vector', \n",
|
||||
" \"FLAT\", \n",
|
||||
" { \"TYPE\": 'FLOAT32', \n",
|
||||
" \"DIM\": 1536, \n",
|
||||
" \"DISTANCE_METRIC\": \"COSINE\"\n",
|
||||
" }, as_name='vector' ),\n",
|
||||
" TextField('$.content', as_name='content')\n",
|
||||
" ]\n",
|
||||
"idx_def = IndexDefinition(index_type=IndexType.JSON, prefix=['doc:'])\n",
|
||||
"client.ft('idx').create_index(schema, definition=idx_def)"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Load Data Files into Redis as JSON Objects with Text and Vector Fields"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 14,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"import os\n",
|
||||
"import openai\n",
|
||||
"\n",
|
||||
"directory = './assets/'\n",
|
||||
"model='text-embedding-ada-002'\n",
|
||||
"i = 1\n",
|
||||
"for file in os.listdir(directory):\n",
|
||||
" with open(os.path.join(directory, file)) as f:\n",
|
||||
" content = f.read()\n",
|
||||
" vector = openai.Embedding.create(input = [content], model = model)['data'][0]['embedding']\n",
|
||||
" client.json().set(f'doc:{i}', '$', {'content': content, 'vector': vector})\n",
|
||||
" i += 1"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Embed the Question and Perform VSS to find the most relevant document"
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 15,
|
||||
"metadata": {},
|
||||
"outputs": [],
|
||||
"source": [
|
||||
"from redis.commands.search.query import Query\n",
|
||||
"import numpy as np\n",
|
||||
"\n",
|
||||
"vec = np.array(openai.Embedding.create(input = [prompt], model = model)['data'][0]['embedding'], dtype=np.float32).tobytes()\n",
|
||||
"q = Query('*=>[KNN 1 @vector $query_vec AS vector_score]')\\\n",
|
||||
" .sort_by('vector_score')\\\n",
|
||||
" .return_fields('content')\\\n",
|
||||
" .dialect(2) \n",
|
||||
"params = {\"query_vec\": vec}\n",
|
||||
"\n",
|
||||
"context = client.ft('idx').search(q, query_params=params).docs[0].content"
|
||||
]
|
||||
},
|
||||
{
|
||||
"attachments": {},
|
||||
"cell_type": "markdown",
|
||||
"metadata": {},
|
||||
"source": [
|
||||
"## Repeat the Question to OpenAI with context\n",
|
||||
"Now that we have relevant context, add that to the prompt to OpenAI and get a very different response."
|
||||
]
|
||||
},
|
||||
{
|
||||
"cell_type": "code",
|
||||
"execution_count": 16,
|
||||
"metadata": {},
|
||||
"outputs": [
|
||||
{
|
||||
"name": "stdout",
|
||||
"output_type": "stream",
|
||||
"text": [
|
||||
"No, Sam Bankman-Fried's company FTX is not considered a well-managed company as it has filed for bankruptcy and owes as much as $8 billion to its creditors. The collapse of FTX has destabilized the crypto industry, and the company is already the target of investigations by the Securities and Exchange Commission and the Justice Department. FTX was widely viewed as one of the most stable and responsible companies in the freewheeling, loosely regulated crypto industry, but its risky practices have become pervasive in crypto, leading to a reckoning.\n"
|
||||
]
|
||||
}
|
||||
],
|
||||
"source": [
|
||||
"prompt = f\"\"\"\n",
|
||||
"Using the information delimited by triple backticks, answer this question: Is Sam Bankman-Fried's company, FTX, considered a well-managed company?\n",
|
||||
"\n",
|
||||
"Context: ```{context}```\n",
|
||||
"\"\"\"\n",
|
||||
"\n",
|
||||
"response = get_completion(prompt)\n",
|
||||
"print(response)"
|
||||
]
|
||||
}
|
||||
],
|
||||
"metadata": {
|
||||
"kernelspec": {
|
||||
"display_name": ".venv",
|
||||
"language": "python",
|
||||
"name": "python3"
|
||||
},
|
||||
"language_info": {
|
||||
"codemirror_mode": {
|
||||
"name": "ipython",
|
||||
"version": 3
|
||||
},
|
||||
"file_extension": ".py",
|
||||
"mimetype": "text/x-python",
|
||||
"name": "python",
|
||||
"nbconvert_exporter": "python",
|
||||
"pygments_lexer": "ipython3",
|
||||
"version": "3.10.6"
|
||||
},
|
||||
"orig_nbformat": 4
|
||||
},
|
||||
"nbformat": 4,
|
||||
"nbformat_minor": 2
|
||||
}
|
||||
@ -0,0 +1,105 @@
|
||||
aiohttp==3.8.4
|
||||
aiosignal==1.3.1
|
||||
anyio==3.6.2
|
||||
argon2-cffi==21.3.0
|
||||
argon2-cffi-bindings==21.2.0
|
||||
arrow==1.2.3
|
||||
asttokens==2.2.1
|
||||
async-timeout==4.0.2
|
||||
attrs==23.1.0
|
||||
backcall==0.2.0
|
||||
beautifulsoup4==4.12.2
|
||||
bleach==6.0.0
|
||||
certifi==2023.5.7
|
||||
cffi==1.15.1
|
||||
charset-normalizer==3.1.0
|
||||
comm==0.1.3
|
||||
debugpy==1.6.7
|
||||
decorator==5.1.1
|
||||
defusedxml==0.7.1
|
||||
et-xmlfile==1.1.0
|
||||
executing==1.2.0
|
||||
fastjsonschema==2.16.3
|
||||
fqdn==1.5.1
|
||||
frozenlist==1.3.3
|
||||
idna==3.4
|
||||
ipykernel==6.22.0
|
||||
ipython==8.13.2
|
||||
ipython-genutils==0.2.0
|
||||
ipywidgets==8.0.6
|
||||
isoduration==20.11.0
|
||||
jedi==0.18.2
|
||||
Jinja2==3.1.2
|
||||
jsonpointer==2.3
|
||||
jsonschema==4.17.3
|
||||
jupyter==1.0.0
|
||||
jupyter-console==6.6.3
|
||||
jupyter-events==0.6.3
|
||||
jupyter_client==8.2.0
|
||||
jupyter_core==5.3.0
|
||||
jupyter_server==2.5.0
|
||||
jupyter_server_terminals==0.4.4
|
||||
jupyterlab-pygments==0.2.2
|
||||
jupyterlab-widgets==3.0.7
|
||||
MarkupSafe==2.1.2
|
||||
matplotlib-inline==0.1.6
|
||||
mistune==2.0.5
|
||||
multidict==6.0.4
|
||||
nbclassic==1.0.0
|
||||
nbclient==0.7.4
|
||||
nbconvert==7.3.1
|
||||
nbformat==5.8.0
|
||||
nest-asyncio==1.5.6
|
||||
notebook==6.5.4
|
||||
notebook_shim==0.2.3
|
||||
numpy==1.24.3
|
||||
openai==0.27.6
|
||||
openpyxl==3.1.2
|
||||
packaging==23.1
|
||||
pandas==2.0.1
|
||||
pandas-stubs==2.0.1.230501
|
||||
pandocfilters==1.5.0
|
||||
parso==0.8.3
|
||||
pexpect==4.8.0
|
||||
pickleshare==0.7.5
|
||||
platformdirs==3.5.0
|
||||
prometheus-client==0.16.0
|
||||
prompt-toolkit==3.0.38
|
||||
psutil==5.9.5
|
||||
ptyprocess==0.7.0
|
||||
pure-eval==0.2.2
|
||||
pycparser==2.21
|
||||
Pygments==2.15.1
|
||||
pyrsistent==0.19.3
|
||||
python-dateutil==2.8.2
|
||||
python-dotenv==1.0.0
|
||||
python-json-logger==2.0.7
|
||||
pytz==2023.3
|
||||
PyYAML==6.0
|
||||
pyzmq==25.0.2
|
||||
qtconsole==5.4.2
|
||||
QtPy==2.3.1
|
||||
redis==4.5.5
|
||||
requests==2.30.0
|
||||
rfc3339-validator==0.1.4
|
||||
rfc3986-validator==0.1.1
|
||||
Send2Trash==1.8.2
|
||||
six==1.16.0
|
||||
sniffio==1.3.0
|
||||
soupsieve==2.4.1
|
||||
stack-data==0.6.2
|
||||
terminado==0.17.1
|
||||
tinycss2==1.2.1
|
||||
tornado==6.3.1
|
||||
tqdm==4.65.0
|
||||
traitlets==5.9.0
|
||||
types-pytz==2023.3.0.0
|
||||
tzdata==2023.3
|
||||
uri-template==1.2.0
|
||||
urllib3==2.0.2
|
||||
wcwidth==0.2.6
|
||||
webcolors==1.13
|
||||
webencodings==0.5.1
|
||||
websocket-client==1.5.1
|
||||
widgetsnbextension==4.0.7
|
||||
yarl==1.9.2
|
||||
Loading…
Reference in New Issue