Price predictions usually combine machine learning and deep learning algorithms, and use historical data and market technical indicators. Here are some common methods:
Intelligent trading systems using reinforcement learning can learn in a simulated market environment and optimize buying and selling decisions over time, gradually improving trading strategies.
AI models can judge market sentiment through sentiment analysis of news and social media and predict stock price changes based on external factors.
Combining multiple algorithms with external data (e.g. economic indicators, company fundamental analysis) often produces the most effective AI stock price prediction models.
When using AI for stock price prediction, the model generates a variety of indicators to help investors make decisions. These indicators can be used to judge market trends and take corresponding trading operations based on different scenarios. Here are some common predictive indicators and strategies for operating them:
This is the future stock price value directly predicted by the model.
A model might predict the probability that a stock's price will rise or fall.
A model may predict a stock's trend direction based on technical indicators such as moving averages.
Technical indicators such as the RSI (relative strength index) can be used to determine whether a stock is overbought or oversold.
The model can predict the future price fluctuation range of stocks.
Forecast based on the strength of buyers and sellers in the market.
The model may suggest reasonable stop loss and take profit points based on historical data.
According to different forecast indicators, investors can make corresponding trading decisions, such as buying, selling or staying on the sidelines. Combining multiple indicators can help make more reasonable investment decisions under different market scenarios.
Among the many rotation judgment indicators, not all indicators have the same predictive power. Based on the market experience in recent years (2020 to 2025), the following is graded according to accuracy and practicality, and explains how to compare across markets and obtain and calculate through process automation.
| index | accuracy rating | Leading time | Verification cases in recent years |
|---|---|---|---|
| Yield curve (2Y-10Y spread) | extremely high | 6 to 18 months | Inversion in July 2022, an accurate warning of regional banking crisis and economic slowdown in 2023 |
| ISM Manufacturing New Orders Index | extremely high | 3 to 6 months | After falling below 42 in 2022Q4, industrial and raw materials stocks bottomed out in 2023Q1 |
| High Yield Bond Spread (HY Spread) | extremely high | 2 to 4 months | In 2022, the interest rate spread will expand to more than 500bp, accurately corresponding to the low point of risk assets |
| Copper/Gold Ratio | high | 2 to 5 months | After the copper-gold ratio bottomed out in March 2020, global cyclical stocks started a year-long rise |
| Global Fund Flows (EPFR Fund Flows) | high | 1 to 3 months | At the end of 2023, funds will flow into emerging market ETFs, leading the rebound of the EM index by about two months. |
| Fed Dot Plot and Futures Implied Interest Rates | high | 3 to 6 months | At the end of 2023, the market is pricing in an interest rate cut in 2024, and growth stocks will start ahead of schedule. |
| PMI differences across countries | high | 1 to 3 months | In 2024, the US PMI will continue to expand while Europe will shrink, and US stocks will continue to outperform European stocks. |
| VIX panic index | medium | Immediately ~ 1 month | Effective as a reverse indicator, but multiple false breakthroughs in 2021 reduce practicality |
| Financing balance and retail investor sentiment survey | medium | Immediately ~ 2 weeks | Extreme values are useful, but there is too much noise in the middle interval, so it needs to be matched with other indicators. |
The reason why the yield curve maintains a high degree of accuracy is because it reflects the bond market's collective pricing of the future economy. The bond market participants are dominated by institutional legal persons, and the information efficiency is much higher than the market dominated by retail investors. ISM's new orders directly reflect the actual demand changes on the enterprise side and are not affected by market sentiment. High-yield bond spreads are the "stress gauge" of the credit market. When corporate default risks increase, funds will be the first to withdraw from high-risk assets. This signal usually precedes the reaction of the stock market.
Relatively speaking, the accuracy of the VIX and financing balances has declined in recent years. The main reason is that zero-day expiration options (0DTE) have significantly changed the option market structure and distorted the VIX; and the collective behavior of retail investors through social media has also reduced the signal quality of traditional sentiment indicators.
When making rotational comparisons between different markets, a comparable standardized framework needs to be established:
Compare the prices of two markets or sectors and observe the trend direction of the ratio. For example, if the "MSCI Emerging Markets Index/MSCI Developed Markets Index" is drawn on a curve, an increase in the ratio means that emerging markets are outperforming. This method can be applied to any two comparable assets:
The indicators of different markets are uniformly converted into Z-Score (multiple of standard deviations from the mean), and horizontal comparisons can only be made after eliminating dimensional differences. The formula is: Z = (current value - mean value of the past N periods) / standard deviation of the past N periods. For example, the U.S. PMI is 52 and the Eurozone PMI is 47. On the surface, there is not much difference. However, if the U.S. PMI historical average is 53, the standard deviation is 3, and the Z-Score is -0.33;
The Citi Economic Surprise Index measures the deviation of actual economic data from market expectations. Comparing this index across countries can determine which region's fundamentals are improving or deteriorating beyond expectations. Funds tend to flow to markets where the surprise index improves.
The following is a complete architecture and program example for building an automated monitoring system using Python.
| Data type | free source | Python suite/API | update frequency |
|---|---|---|---|
| Stock price, ETF price | Yahoo Finance | yfinance | real time/daily line |
| U.S. Treasury bond yields | FRED (Federal Reserve Database) | fredapi | daily |
| PMI、GDP、CPI | FRED / OECD | fredapi / pandas-datareader | month/quarter |
| high yield bond spread | FRED(BAMLH0A0HYM2) | fredapi | daily |
| Copper and gold prices | Yahoo Finance | yfinance(HG=F, GC=F) | real time/daily line |
| VIX index | Yahoo Finance / CBOE | yfinance(^VIX) | immediate |
| Fund flow | Estimated changes in ETF holdings | yfinance (trading volume + change in net worth) | daily |
| Futures Implied Interest Rate | CME FedWatch (crawler required) | requests + BeautifulSoup | immediate |
# === Install required packages === # pip install yfinance fredapi pandas numpy schedule import yfinance as yf import pandas as pd import numpy as np from fredapi import Fred from datetime import datetime, timedelta # FRED API Key (go to https://fred.stlouisfed.org/docs/api/api_key.html to apply for free) fred = Fred(api_key='YOUR_FRED_API_KEY') # ========================================== # 1. Yield curve: 2-year-10-year interest rate spread # ========================================== def get_yield_curve_spread(): gs10 = fred.get_series('DGS10') # 10-Year Treasury Bond Yield gs2 = fred.get_series('DGS2') # 2-Year Treasury Bond Yield spread = gs10 - gs2 spread = spread.dropna() latest = spread.iloc[-1] status = "Upside Down (Recession Warning)" if latest < 0 else "normal" return { 'spread': round(latest, 3), 'status': status, 'series': spread.tail(252) # Information for the past year } # ========================================== # 2. High Yield Bond Spread # ========================================== def get_hy_spread(): hy = fred.get_series('BAMLH0A0HYM2') hy = hy.dropna() latest = hy.iloc[-1] avg_1y = hy.tail(252).mean() z_score = (latest - hy.tail(756).mean()) / hy.tail(756).std() return { 'spread_bp': round(latest * 100, 0), 'z_score': round(z_score, 2), 'risk_level': "high risk" if z_score > 1.5 else "neutral" if z_score > -0.5 else "low risk" } # ========================================== # 3. Bronze-gold ratio # ========================================== def get_copper_gold_ratio(): copper = yf.download('HG=F', period='2y')['Close'] gold = yf.download('GC=F', period='2y')['Close'] ratio = copper / gold ratio = ratio.dropna() current = ratio.iloc[-1] ma_200 = ratio.rolling(200).mean().iloc[-1] trend = "Signal of economic expansion" if current > ma_200 else "Signal of economic contraction" return {'ratio': round(current, 5), 'ma200': round(ma_200, 5), 'trend': trend} # ========================================== # 4. Comparison of relative strength across markets # ========================================== def relative_strength(ticker_a, ticker_b, period='1y'): """Calculate the relative strength ratio and trend of two assets""" a = yf.download(ticker_a, period=period)['Close'] b = yf.download(ticker_b, period=period)['Close'] ratio = a / b ratio = ratio.dropna() ma_50 = ratio.rolling(50).mean() latest_ratio = ratio.iloc[-1] latest_ma = ma_50.iloc[-1] outperformer = ticker_a if latest_ratio > latest_ma else ticker_b return { 'ratio': round(latest_ratio, 4), 'ma50': round(latest_ma, 4), 'outperformer': outperformer, 'series': ratio } #Usage example: # relative_strength('IWF', 'IWD') # Growth vs value # relative_strength('EEM', 'SPY') # Emerging markets vs US stocks # relative_strength('^TWII', '000300.SS') # Taiwan stocks vs CSI 300 # ========================================== # 5. Z-Score Normalized Cross-Market Comparison # ========================================== def zscore_compare(series_dict, lookback=756): """ Receive time series from multiple markets, calculate Z-Score and compare horizontally series_dict: {'United States': pd.Series, 'Europe': pd.Series, ...} """ results = {} for name, series in series_dict.items(): s = series.dropna().tail(lookback) current = s.iloc[-1] z = (current - s.mean()) / s.std() results[name] = { 'current': round(current, 3), 'z_score': round(z, 2), 'percentile': round((s < current).mean() * 100, 1) } return pd.DataFrame(results).T.sort_values('z_score', ascending=False) # ========================================== # 6. Sector Momentum Ranking # ========================================== def sector_momentum_ranking(): """Obtained multi-period momentum and ranking of 11 major U.S. stock sector ETFs""" sectors = { 'science and technology': 'XLK', 'finance': 'XLF', 'Medical': 'XLV', 'Non-essential consumption': 'XLY', 'Essential consumption': 'XLP', 'industry': 'XLI', 'energy': 'XLE', 'Raw materials': 'XLB', 'Utilities': 'XLU', 'real estate': 'XLRE', 'communication': 'XLC' } results = [] for name, ticker in sectors.items(): data = yf.download(ticker, period='1y')['Close'] ret_1m = (data.iloc[-1] / data.iloc[-21] - 1) * 100 ret_3m = (data.iloc[-1] / data.iloc[-63] - 1) * 100 ret_6m = (data.iloc[-1] / data.iloc[-126] - 1) * 100 # Comprehensive momentum score: higher weight in the near future score = ret_1m * 0.4 + ret_3m * 0.35 + ret_6m * 0.25 results.append({ 'Plate': name, '1 month%': round(ret_1m, 2), '3 months%': round(ret_3m, 2), '6 months%': round(ret_6m, 2), 'Comprehensive kinetic energy': round(score, 2) }) df = pd.DataFrame(results).sort_values('Comprehensive kinetic energy', ascending=False) return df.reset_index(drop=True)
import schedule import time import requests def send_line_notify(token, msg): """Send a message via LINE Notify""" headers = {'Authorization': f'Bearer {token}'} requests.post('https://notify-api.line.me/api/notify', headers=headers, data={'message': msg}) def daily_rotation_report(): """Daily rotation monitoring report""" yc = get_yield_curve_spread() hy = get_hy_spread() cg = get_copper_gold_ratio() sectors = sector_momentum_ranking() report = f""" === Daily rotation monitoring === Date: {datetime.now().strftime('%Y-%m-%d')} Yield curve (2Y-10Y): {yc['spread']}% {yc['status']} High Yield Bond Spread Z-Score: {hy['z_score']} ({hy['risk_level']}) Copper-gold ratio trend: {cg['trend']} Top three sector momentum rankings: {sectors.head(3).to_string(index=False)} Sector momentum ranks bottom three: {sectors.tail(3).to_string(index=False)} """ print(report) # send_line_notify('YOUR_LINE_TOKEN', report) return report # Executed at 6pm every trading day schedule.every().monday.at("18:00").do(daily_rotation_report) schedule.every().tuesday.at("18:00").do(daily_rotation_report) schedule.every().wednesday.at("18:00").do(daily_rotation_report) schedule.every().thursday.at("18:00").do(daily_rotation_report) schedule.every().friday.at("18:00").do(daily_rotation_report) while True: schedule.run_pending() time.sleep(60)
def rotation_dashboard(): """ Comprehensive multiple indicators to output judgment on the current economic cycle stage Return: Probability estimate of recovery/expansion/overheating/recession """ scores = {'recovery': 0, 'expansion': 0, 'overheat': 0, 'decline': 0} #Yield Curve Signal yc = get_yield_curve_spread() if yc['spread'] < -0.5: scores['decline'] += 3 elif yc['spread'] < 0: scores['overheat'] += 2; scores['decline'] += 1 elif yc['spread'] < 1.0: scores['expansion'] += 2 else: scores['recovery'] += 3 # High Yield Bond Spread Signal hy = get_hy_spread() if hy['z_score'] > 1.5: scores['decline'] += 3 elif hy['z_score'] > 0.5: scores['overheat'] += 2 elif hy['z_score'] > -0.5: scores['expansion'] += 2 else: scores['recovery'] += 2 # copper-gold ratio signal cg = get_copper_gold_ratio() if cg['ratio'] > cg['ma200']: scores['expansion'] += 2 else: scores['decline'] += 1; scores['recovery'] += 1 # Convert to probability distribution total = sum(scores.values()) probs = {k: round(v/total*100, 1) for k, v in scores.items()} phase = max(probs, key=probs.get) return { 'Judgment cycle': phase, 'Probability of each stage': probs, 'Recommended configuration': { 'recovery': 'Overweight technology and finance; underweight defensive', 'expansion': 'Overweight raw materials and energy; maintain high water level of stocks', 'overheat': 'Underweight growth stocks; increase commodities and anti-inflation assets', 'decline': 'Increase public debt and cash; overweight defensive sectors' }[phase] }
def tw_sector_rotation(): """Taiwan stock ETF momentum tracking""" tw_sectors = { 'Taiwan Semiconductor': '00891.TW', 'Taiwan ESG Sustainability': '00850.TW', 'Taiwan Finance': '0055.TW', 'Taiwan High Dividends': '0056.TW', 'Taiwan 50': '0050.TW', 'Taiwan Medium 100': '0051.TW', } results = [] for name, ticker in tw_sectors.items(): try: data = yf.download(ticker, period='6mo')['Close'].dropna() if len(data) < 63: continue ret_1m = (data.iloc[-1] / data.iloc[-21] - 1) * 100 ret_3m = (data.iloc[-1] / data.iloc[-63] - 1) * 100 results.append({ 'Stock': name, '1 Monthly Remuneration%': round(ret_1m, 2), '3 monthly remuneration%': round(ret_3m, 2), 'Kinetic energy fraction': round(ret_1m * 0.5 + ret_3m * 0.5, 2) }) except Exception as e: print(f"{name}Failed to obtain:{e}") return pd.DataFrame(results).sort_values('Kinetic energy fraction', ascending=False)
The core value of programmed monitoring is not to replace human judgment, but to eliminate emotional interference and ensure discipline. The following points need special attention when building the system:
yfinance is an open source Python suite developed by Ran Aroussi for obtaining financial market data from Yahoo Finance. After Yahoo Finance closed its official API in 2017, yfinance became the most popular tool for accessing its public information. The latest version is 1.2.0 (released in February 2026), which is licensed by Apache and is completely free and suitable for research and educational purposes.
# Install pip install yfinance #Upgrade to the latest version pip install yfinance --upgrade # Basic import import yfinance as yf
yfinance's dependency packages include pandas, numpy, requests and lxml, all of which are pre-installed if using the Anaconda environment.
| Class/Function | use | Usage examples |
|---|---|---|
| Ticker | Access all data on a single target (prices, financial reports, dividends, etc.) | yf.Ticker("AAPL") |
| Tickers | Process multiple targets simultaneously | yf.Tickers("AAPL MSFT GOOG") |
| download() | Download historical prices of multiple targets in batches (the most commonly used function) | yf.download("SPY QQQ", period="1y") |
| Search | search code | yf.Search("TSMC") |
| Market | Access market summary data | yf.Market("us_market") |
| Sector / Industry | Access industry and sector information | yf.Sector("technology") |
| WebSocket | Live streaming market data (v1.0+ new features) | yf.WebSocket(on_message=callback) |
The information yfinance can obtain goes far beyond simple stock prices, covering the following categories:
| Data category | Specific content | Corresponding properties/methods |
|---|---|---|
| historical price | OHLCV (open high low close volume), adjusted closing price | .history() or yf.download() |
| Basic company information | Market capitalization, price-to-earnings ratio, 52-week high and low, industry classification, number of employees, etc. | .info |
| financial statements | Profit and loss statement, balance sheet, cash flow statement (annual and quarterly) | .income_stmt / .balance_sheet / .cashflow / .quarterly_income_stmt |
| Dividends and Splits | Historical dividend distribution and stock split records | .dividends / .splits / .actions |
| Analyst information | Target price, rating, earnings estimate | .analyst_price_targets / .recommendations |
| option | Expiration date, call/put chain | .options / .option_chain() |
| Legal person holdings | Major institutional holdings, insider trading | .institutional_holders / .insider_transactions |
| calendar | Financial report date, ex-dividend date, etc. | .calendar |
| market type | Code format | example |
|---|---|---|
| US stocks | Enter the code directly | AAPL, MSFT, TSLA, SPY |
| Taiwan stocks | Code .TW (listed) / Code .TWO (listed) | 2330.TW (TSMC), 0050.TW, 0056.TW |
| Japanese stocks | Code.T | 7203.T(Toyota), 6758.T(Sony) |
| Hong Kong stocks | Code.HK | 0700.HK (Tencent), 9988.HK (Alibaba) |
| Mainland stocks | Code.SS(Shanghai)/Code.SZ(Shenzhen) | 600519.SS (Maotai), 000001.SZ |
| European stocks | Code.Exchange suffix | SAP.DE (Germany), MC.PA (France LVMH), AZN.L (London) |
| index | Starting with ^ | ^GSPC (S&P 500), ^DJI (Dow Jones), ^IXIC (Nasdaq), ^TWII (Weighted Index), ^N225 (Nikkei) |
| futures | Code=F | GC=F (gold), SI=F (silver), CL=F (crude oil), HG=F (copper), NG=F (natural gas) |
| cryptocurrency | Code-USD | BTC-USD, ETH-USD, SOL-USD, ADA-USD |
| Forex | Code 1 Code 2 = X | EURUSD=X, JPYUSD=X, TWDUSD=X |
| ETF | Enter the code directly | SPY, QQQ, EEM, VGK, EWT, EWJ |
import yfinance as yf # ======================================== # Method 1: Use the Ticker object (suitable for in-depth analysis of a single target) # ======================================== tsmc = yf.Ticker("2330.TW") # Get historical prices hist = tsmc.history(period="1y") # nearly a year hist = tsmc.history(period="6mo") # Last 6 months hist = tsmc.history(period="5d") # Last 5 days hist = tsmc.history(period="max") # all history hist = tsmc.history(start="2024-01-01", end="2025-12-31") #Specify date range # period parameter options: # 1d, 5d, 1mo, 3mo, 6mo, 1y, 2y, 5y, 10y, ytd, max # ======================================== #Method 2: Use download() (suitable for batch downloading of multiple standards) # ======================================== data = yf.download( tickers="SPY QQQ EEM GC=F BTC-USD", period="1y", interval="1d", #日线 group_by="ticker", #Group by target auto_adjust=True, # Automatically adjust ex-rights interest threads=True #Multiple threads accelerate downloading ) # Access specific objects: data['SPY']['Close'] # ======================================== # Time granularity (interval parameter) # ======================================== # Minute level: 1m, 2m, 5m, 15m, 30m, 60m, 90m # Hours: 1h # Day level and above: 1d, 5d, 1wk, 1mo, 3mo # # Note restrictions: # 1m data can only be retrieved from the last 7 days # Intraday data (interval < 1d) can only take the last 60 days # Above the daily line, decades of history are available # Get the 5-minute K-line (last 5 days) intraday = yf.download("AAPL", period="5d", interval="5m")
Both history() and download() return pandas DataFrame, which contains the following fields:
| field | illustrate |
|---|---|
| Open | opening price |
| High | highest price |
| Low | lowest price |
| Close | Closing price (default adjusted for ex-dividends) |
| Volume | Volume |
| Dividends | Dividends (only .history() has) |
| Stock Splits | Stock splits (only available with .history()) |
msft = yf.Ticker("MSFT") # === Basic company information (return dict) === info = msft.info print(info['marketCap']) # market capitalization print(info['trailingPE']) # Price to Earning Ratio print(info['dividendYield']) # Dividend Yield print(info['fiftyTwoWeekHigh']) #52 Week High print(info['sector']) #industry print(info['longBusinessSummary'])# Company Profile # === Financial statements (return DataFrame) === msft.income_stmt # Annual Profit and Loss Statement msft.quarterly_income_stmt # Quarterly Profit and Loss Statement msft.balance_sheet # Annual Balance Sheet msft.quarterly_balance_sheet # Quarterly Balance Sheet msft.cashflow #Annual Cash Flow Statement msft.quarterly_cashflow # Quarterly Cash Flow Statement # === Dividends and Splits === msft.dividends #Historical Dividends msft.splits # Historical stock splits msft.actions # Dividend + Split Merger # === Analyst Information === msft.analyst_price_targets # Target price (high/low/average/median) msft.recommendations #Analyst Ratings msft.calendar # Calendar (financial report day, etc.) # === Legal person shareholding === msft.institutional_holders # Institutional holdings msft.major_holders # Major shareholders msft.insider_transactions #Insider Trading # === Choice === msft.options # List of available due dates chain = msft.option_chain(msft.options[0]) chain.calls # Call information chain.puts # Put information
# ======================================== # 1. Obtain the closing price of a single target after batch downloading # ======================================== data = yf.download(["SPY", "GC=F", "BTC-USD"], period="1y") spy_close = data['Close']['SPY'] # Multi-index is MultiIndex # Direct access when downloading a single target spy = yf.download("SPY", period="1y") spy_close = spy['Close'] # ======================================== # 2. .squeeze() technique for handling MultiIndex # ======================================== # When downloading a single target, the Close field may be DataFrame instead of Series. # Use .squeeze() to ensure conversion to Series close = yf.download("AAPL", period="1y")['Close'].squeeze() # ======================================== # 3. Error handling (yfinance may fail occasionally) # ======================================== def safe_download(ticker, **kwargs): """Safe download, return empty DataFrame when failed""" try: data = yf.download(ticker, progress=False, **kwargs) if data.empty: print(f"{ticker}: No information") return data except Exception as e: print(f"{ticker}Download failed:{e}") return pd.DataFrame() # ======================================== # 4. Quick check of commonly used codes of Taiwan stocks # ======================================== tw_tickers = { 'TSMC': '2330.TW', 'Hon Hai': '2317.TW', 'MediaTek': '2454.TW', 'Taiwan 50': '0050.TW', 'High dividend': '0056.TW', 'finance': '0055.TW', 'semiconductor': '00891.TW', 'Weighted index': '^TWII', } # ======================================== # 5. Complete example of calculating technical indicators # ======================================== import pandas as pd ticker = yf.Ticker("2330.TW") df = ticker.history(period="1y") # moving average df['MA20'] = df['Close'].rolling(20).mean() df['MA60'] = df['Close'].rolling(60).mean() df['EMA12'] = df['Close'].ewm(span=12).mean() # RSI delta = df['Close'].diff() gain = delta.where(delta > 0, 0).rolling(14).mean() loss = (-delta.where(delta < 0, 0)).rolling(14).mean() df['RSI'] = 100 - (100 / (1 + gain / loss)) # Bollinger Bands df['BB_mid'] = df['Close'].rolling(20).mean() df['BB_std'] = df['Close'].rolling(20).std() df['BB_upper'] = df['BB_mid'] + 2 * df['BB_std'] df['BB_lower'] = df['BB_mid'] - 2 * df['BB_std'] print(df[['Close','MA20','RSI','BB_upper','BB_lower']].tail()) # ======================================== # 6. Export to CSV # ======================================== df.to_csv("tsmc_data.csv", encoding="utf-8-sig") #utf-8-sig allows Excel to display Chinese correctly
| limit | Detailed description | coping strategies |
|---|---|---|
| Unofficial API | yfinance is not affiliated with Yahoo. It uses Yahoo's public API and may be temporarily invalid due to Yahoo version revision. | Keep updated to the latest version; prepare backup data sources (for example, FinMind can be used for Taiwan stocks) |
| frequency limit | A large number of requests in a short period of time may cause the IP to be temporarily blocked by Yahoo | Add time.sleep(1) to the loop; use progress=False to reduce requests; cache data locally |
| Intraday data deadline | 1-minute data is only retained for 7 days; intra-day data is kept for up to 60 days | If long-term intraday data is required, it should be downloaded regularly and stored in the local database. |
| Data quality | Occasionally there are missing values or outliers; some unpopular targets have incomplete data | Be sure to use .dropna() to clean up after downloading; use multi-source cross-validation for important analysis |
| For personal/research use only | Yahoo Finance's terms of use limit material to personal use and not for commercial resale. | Commercial use should use paid APIs (such as Bloomberg, Refinitiv, Polygon.io) |
| Some methods use web crawlers | A few functions (such as some .info fields) are obtained by crawling Yahoo web pages, which is relatively unstable. | The core .history() and download() use formal APIs and are highly stable. |
| Kits/Services | cost | Advantages | Disadvantages |
|---|---|---|---|
| yfinance | free | Easy to use, wide range of information, and active community | Unofficial, occasionally invalid, not suitable for high-frequency trading |
| Alpha Vantage | Free (limited) / Paid | Built-in technical indicator calculation and official API Key | Free version has a limit of 5 requests per minute |
| FRED API(fredapi) | free | The most authoritative general economic data (yield rate, PMI, GDP, etc.) | Only general economic data, no individual stock prices |
| FinMind | Free (limited) | The most complete information on Taiwan stocks, including legal person chips, financing bonds, etc. | Only covers Taiwan market |
| Polygon.io | Free (delayed) / paid (instant) | Extremely low latency (1ms), suitable for real-time transactions | Free version data is delayed by 15 minutes |
For personal research and learning, strategy prototyping, and mid- to long-term investment analysis, yfinance is the best place to start. Its simple design allows beginners to obtain global market data with two or three lines of code and analyze it with pandas. When the needs are upgraded to real-time transactions or commercial applications, you can then turn to paid APIs.
MetaQuotes Language (MQL for short) is a programming language developed specifically for financial market trading and is used to create automated trading strategies (Expert Advisors), custom indicators, scripts and function libraries in MetaTrader platforms (such as MT4 and MT5).
// Operate every time a new K stick starts
int start() {
if (OrdersTotal() == 0 && Volume[0] == 1) {
OrderSend(Symbol(), OP_BUY, 0.1, Ask, 3, 0, 0, "Buy Order", 0, 0, clrGreen);
}
return 0;
}
#property indicator_separate_window
#property indicator_buffers 1
double Buffer[];
int OnInit() {
SetIndexBuffer(0, Buffer);
return(INIT_SUCCEEDED);
}
int OnCalculate(const int rates_total, const int prev_calculated,
const datetime &time[], const double &open[],
const double &high[], const double &low[],
const double &close[], const long &tick_volume[],
const long &volume[], const int &spread[]) {
for (int i = 0; i < rates_total; i++) {
Buffer[i] = close[i] - open[i];
}
return(rates_total);
}
MQL code can be edited and compiled through MetaTrader's built-in MetaEditor, and backtested and optimized in the strategy tester.
MetaQuotes Language is a professional language created for trading automation. Both beginners and professional quantitative traders can use its powerful functions to implement sophisticated trading strategies.
This is the most complete version, including multiple indicators, drawing tools and a complete trading interface (if connected to a broker).
<!-- Advanced chart container -->
<div class="tradingview-widget-container">
<div id="tradingview_adv"></div>
<script type="text/javascript" src="https://s3.tradingview.com/tv.js"></script>
<script type="text/javascript">
new TradingView.widget({
"autosize": true,
"symbol": "BINANCE:BTCUSDT",
"interval": "H",
"timezone": "Etc/UTC",
"theme": "dark",
"style": "1",
"locale": "zh_TW",
"container_id": "tradingview_adv"
});
</script>
</div>
This tool is presented as a dashboard and automatically calculates and displays buy or sell recommendations based on a variety of technical indicators (e.g. moving averages, oscillators).
<!-- Technical Analysis Gadget -->
<div class="tradingview-widget-container">
<div class="tradingview-widget-container__widget"></div>
<script type="text/javascript" src="https://s3.tradingview.com/external-embedding/embed-widget-technical-analysis.js" async>
{
"interval": "1m",
"width": 425,
"isTransparent": false,
"height": 450,
"symbol": "NASDAQ:TSLA",
"showIntervalTabs": true,
"locale": "zh_TW",
"colorTheme": "light"
}
</script>
</div>
It is suitable for displaying real-time price comparisons of multiple products. It is often found in the sidebar of the homepage of financial websites.
<!-- Market Overview -->
<div class="tradingview-widget-container">
<script type="text/javascript" src="https://s3.tradingview.com/external-embedding/embed-widget-market-overview.js" async>
{
"colorTheme": "dark",
"dateRange": "12M",
"showChart": true,
"locale": "zh_TW",
"width": "100%",
"height": "400",
"tabs": [
{
"title": "index",
"symbols": [
{"s": "FOREXCOM:SPX500", "d": "S&P 500"},
{"s": "FOREXCOM:NSXUSD", "d": "Nasdaq 100"}
]
}
]
}
</script>
</div>
Although it is possible to write JavaScript manually, TradingView provides an official graphical generator, which is recommended to avoid syntax errors:
| question | Solution |
|---|---|
| Chart cannot be loaded | examinecontainer_idWhether it exactly matches the HTML ID. |
| width running version | Willwidthset to"100%"And make sure the outer div has a fixed width. |
| data delay | Free version widget data is typically delayed by 15-20 minutes, depending on the exchange. |
Binance is a leading global cryptocurrency exchange that provides developers with rich API support, includingSpot APIandClient API, to facilitate users to conduct automatic transactions and data acquisition.
Binance Spot API is an API designed by Binance for spot market traders. It can be used to query market information, place orders, cancel orders and other operations. This API is commonly used to design trading bots, automated trading strategies, and monitor market fluctuations.
The Binance Client API provides a convenient way to access Binance’s various API methods. Developers can usebinance.clientlibrary API
Authentication and management, and conveniently call functions of various spot and contract markets.
pip install binancebinance.clientMake an API connection and callbinance.spotmethod.from binance.client import Client
#Initialize client
client = Client(api_key='your_api_key', api_secret='your_secret_key')
# Get the current price
price = client.get_symbol_ticker(symbol="BTCUSDT")
print(price)
Bybit provides REST and WebSocket APIs, which can be used to query market conditions, place orders, check funding rates, manage accounts, etc. The following shows how to use PythonrequestsThe suite calls Bybit's public API.
pip install requests
import requests
BASE_URL = "https://api.bybit.com"
def get_symbols():
url = f"{BASE_URL}/v5/market/instruments-info?category=linear"
res = requests.get(url)
res.raise_for_status()
data = res.json()
symbols = [s["symbol"] for s in data["result"]["list"]]
print(f"A total of {len(symbols)} trading pairs obtained:")
print(symbols[:10]) # Display the first 10 items
if __name__ == "__main__":
get_symbols()
import requests
def get_klines(symbol="BTCUSDT", interval="60", limit=5):
url = f"{BASE_URL}/v5/market/kline?category=linear&symbol={symbol}&interval={interval}&limit={limit}"
res = requests.get(url)
res.raise_for_status()
data = res.json()
for k in data["result"]["list"]:
print(k)
if __name__ == "__main__":
get_klines()
Bybit’s private endpoints such as order placement and asset query require API Key and signature.
import requests, time, hmac, hashlib
API_KEY = "your_api_key"
API_SECRET = "your_api_secret"
def sign_request(params, secret):
"""Bybit signature generation"""
query = "&".join([f"{k}={v}" for k, v in sorted(params.items())])
return hmac.new(secret.encode(), query.encode(), hashlib.sha256).hexdigest()
def get_wallet_balance():
endpoint = "/v5/account/wallet-balance"
url = BASE_URL + endpoint
timestamp = str(int(time.time() * 1000))
params = {
"accountType": "UNIFIED",
"timestamp": timestamp,
"api_key": API_KEY,
}
params["sign"] = sign_request(params, API_SECRET)
res = requests.get(url, params=params)
print(res.json())
if __name__ == "__main__":
get_wallet_balance()
Pionex uses/api/v1/common/symbols?type=PERPTo obtain the "perpetual contract" trading pair; on Bybit, you can use/v5/market/instruments-infoand specifycategory=linear(USDT perpetual) orinverse(reverse contract) achieves the same effect.
import requests
classBybitAPI:
BASE_URL = "https://api.bybit.com"
@classmethod
def get_symbols(cls, category="linear"):
"""
Get a specific type of trading pair
category can be:
- linear → USDT Perpetual (PERP)
- inverse → Inverse perpetual/delivery contract
- spot → spot
"""
endpoint = "/v5/market/instruments-info"
url = f"{cls.BASE_URL}{endpoint}"
params = {"category": category}
res = requests.get(url, params=params)
res.raise_for_status()
data = res.json()
if data.get("retCode") == 0:
symbols = [s["symbol"] for s in data["result"]["list"]]
print(f"A total of {len(symbols)} {category} type trading pairs obtained")
for s in symbols[:10]:
print(s)
else:
print("Acquisition failed:", data)
if __name__ == "__main__":
BybitAPI.get_symbols("linear")
| Pionex | Bybit | illustrate |
|---|---|---|
type=PERP | category=linear | USDT Perpetual Contract |
type=SPOT | category=spot | spot market |
| — | category=inverse | Inverse perpetual or delivery contracts |
{
"retCode": 0,
"result": {
"list": [
{
"symbol": "BTCUSDT",
"contractType": "LinearPerpetual",
"status": "Trading",
"lotSizeFilter": {
"minOrderQty": "0.001",
"maxOrderQty": "100",
"qtyStep": "0.001"
},
"priceFilter": {
"tickSize": "0.5"
}
}
]
}
}
category=linearThat is, it corresponds to the "PERP" trading pair of Pionex.priceFilter、lotSizeFilterand other restrictions.Available in Pionex/api/v1/market/klinesCheck the market situation; the corresponding endpoint of Bybit is/v5/market/kline. throughcategorySpecify the market type (e.g.linearRepresents USDT perpetual contract) and can be passed insymbol、interval、limit、endTimeand other parameters.
import requests
import time
classBybitAPI:
BASE_URL = "https://api.bybit.com"
@classmethod
def get_klines(cls, symbol: str, interval: str, end_time: int = None, limit: int = 100):
"""
Query Bybit K-line information
:param symbol: trading pair, such as "BTCUSDT"
:param interval: time interval (1, 3, 5, 15, 30, 60, 120, 240, 360, 720, D, W, M)
:param end_time: End time (Unix milliseconds), default is now
:param limit: Number of returned transactions, maximum 1000
"""
endpoint = "/v5/market/kline"
url = f"{cls.BASE_URL}{endpoint}"
params = {
"category": "linear", # USDT Perpetual
"symbol": symbol,
"interval": interval,
"limit": limit
}
if end_time:
params["end"] = end_time
else:
params["end"] = int(time.time() * 1000)
res = requests.get(url, params=params)
res.raise_for_status()
data = res.json()
if data.get("retCode") == 0:
klines = data["result"]["list"]
print(f"{symbol} obtained a total of {len(klines)} K lines")
# Display the first 3 root information
for k in klines[:3]:
open_time, open_price, high, low, close, volume, turnover = k
print(f"Open:{open_price} Close:{close} High:{high} Low:{low} Volume:{volume}")
else:
print("Acquisition failed:", data)
if __name__ == "__main__":
BybitAPI.get_klines(symbol="BTCUSDT", interval="60", limit=5)
{
"retCode": 0,
"result": {
"symbol": "BTCUSDT",
"category": "linear",
"list": [
[
"1735119600000", // start time (milliseconds)
"98342.5", // opening price
"98350.0", // highest price
"98285.0", // lowest price
"98290.5", // closing price
"12.304", // trading volume
"1210000.5" // Transaction volume (USDT)
]
]
}
}
categoryCan be changed tospotorinverseCorrespond to different markets.endThe argument is a timestamp in milliseconds (Unix epoch × 1000).Below is a directly reproducible Python example using a ThreadPoolExecutor with built-in retries, rate limiting, and cache. Process: First load the captured K-line (if any) from the cache, and only issue requests for missing or insufficient trading pairs; use thread pool to execute multiple requests at the same time and use semaphore to control the number of concurrency to avoid being limited; in case of failure, exponential backoff will be used to retry.
# Requirements: pip install requests
import requests
import time
import json
import os
from concurrent.futures import ThreadPoolExecutor, as_completed
from threading import Semaphore
BASE_URL = "https://api.bybit.com/v5/market/kline"
CACHE_FILE = "klines_cache.json"
# Adjustable parameters
MAX_WORKERS = 20 # Thread pool size (subject to API rate limit adjustment)
MAX_CONCURRENT = 10 # Number of real concurrent requests (controlled with semaphore)
RETRY = 3 # Number of retries for each request
INITIAL_BACKOFF = 0.5 # Number of seconds to wait for the first retry
CATEGORY = "linear" # linear -> USDT sustainable; spot/inverse can be changed
LIMIT_PER_CALL = 200 # Each kline API limit (depending on the API upper limit setting)
DATASET_ALLDAYS = 24 * 6 # Example: Determine at least how many K lines are needed (can be changed)
# Load/access cache
def load_cache():
if os.path.exists(CACHE_FILE):
try:
return json.load(open(CACHE_FILE, "r", encoding="utf-8"))
exceptException:
return {}
return {}
def save_cache(cache):
json.dump(cache, open(CACHE_FILE, "w", encoding="utf-8"), ensure_ascii=False, indent=2)
# API capture of a single symbol, including retry and rate control (the number of simultaneous executions is controlled by semaphore)
def fetch_klines_for_symbol(symbol, interval="60", end_time=None, limit=LIMIT_PER_CALL, sem: Semaphore = None):
params = {
"category": CATEGORY,
"symbol": symbol,
"interval": interval,
"limit": limit
}
if end_time:
params["end"] = int(end_time)
backoff = INITIAL_BACKOFF
last_exc = None
# Get semaphore (if provided)
if sem:
sem.acquire()
try:
for attempt in range(1, RETRY + 1):
try:
resp = requests.get(BASE_URL, params=params, timeout=10)
resp.raise_for_status()
data = resp.json()
# Bybit v5 returns retCode == 0 to indicate success.
if data.get("retCode", 0) == 0 and "result" in data:
klines = data["result"].get("list", [])
return klines
else:
last_exc = Exception(f"API error: {data}")
except requests.exceptions.RequestException as e:
last_exc = e
#backoff
time.sleep(backoff)
backoff *= 2
finally:
if sem:
sem.release()
# If failed, throw the last error or return None
raise last_exc
# Batch processing: pass in pairs (list of symbols), return dict {symbol: klines}
def get_klines_batch(pairs, interval="60", dataset_alldays=DATASET_ALLDAYS, limit=LIMIT_PER_CALL, max_workers=MAX_WORKERS, max_concurrent=MAX_CONCURRENT):
cache = load_cache() # cache format: { symbol: [kline_list] }
results = {}
to_fetch = []
# Determine which symbols need to be grabbed (does not exist in cache or is not long enough)
for s in pairs:
cached = cache.get(s)
if cached and len(cached) >= dataset_alldays:
results[s] = cached
else:
to_fetch.append(s)
# If there is nothing to fetch, just return it directly
if not to_fetch:
return results
sem = Semaphore(max_concurrent)
with ThreadPoolExecutor(max_workers=max_workers) as exe:
futures = {exe.submit(fetch_klines_for_symbol, sym, interval, None, limit, sem): sym for sym in to_fetch}
for fut in as_completed(futures):
sym = futures[fut]
try:
kl = fut.result()
# If the API return is stored in list format (according to Bybit example [time,open,high,low,close,vol,turnover])
cache[sym] = kl
results[sym] = kl
except Exception as e:
# Record failure, but do not block the entire program
print(f"[Error] Failed to obtain {sym}: {e}")
results[sym] = None
# Archive the cache (optional: only save successful ones)
save_cache(cache)
return results
# Example usage
if __name__ == "__main__":
# Assume there are 500 pairs (indicative)
pairs = ["BTCUSDT", "ETHUSDT", "SOLUSDT"] # ... 500
# Execute batch fetching
all_klines = get_klines_batch(pairs, interval="60", dataset_alldays=100, limit=200)
# Filter out symbols that meet the length requirements
good = [s for s, kl in all_klines.items() if kl and len(kl) >= 100]
print(f"The number of trading pairs matching more than 100: {len(good)}")
print(good[:20])
MAX_WORKERSandMAX_CONCURRENT. If you have an API key, you can usually increase the rate quota.Pionex provides an official API that allows developers to automate transactions, query market data, and manage account assets through programs. The API supports both REST and WebSocket methods.
API KeyandSecret, shown only once// Use Node.js axios to request the Pionex API
const axios = require("axios");
const crypto = require("crypto");
const apiKey = "Your API_KEY";
const secret = "Your API_SECRET";
const baseUrl = "https://api.pionex.com";
// Signature generation
function sign(query) {
return crypto.createHmac("sha256", secret).update(query).digest("hex");
}
// Query account balance
async function getBalances() {
const timestamp = Date.now();
const query = `timestamp=${timestamp}`;
const signature = sign(query);
const res = await axios.get(`${baseUrl}/api/v1/account?${query}&signature=${signature}`, {
headers: { "X-MBX-APIKEY": apiKey }
});
console.log(res.data);
}
getBalances();
import time
import hmac
import hashlib
import requests
API_KEY = "Your API_KEY"
SECRET = "yourAPI_SECRET"
BASE_URL = "https://api.pionex.com"
def sign(query: str) -> str:
return hmac.new(SECRET.encode(), query.encode(), hashlib.sha256).hexdigest()
def get_balances():
timestamp = str(int(time.time() * 1000))
query = f"timestamp={timestamp}"
signature = sign(query)
url = f"{BASE_URL}/api/v1/account?{query}&signature={signature}"
headers = {"X-MBX-APIKEY": API_KEY}
res = requests.get(url, headers=headers)
print(res.json())
get_balances()
const WebSocket = require("ws");
const ws = new WebSocket("wss://ws.pionex.com/ws");
ws.on("open", () => {
console.log("Connected to Pionex WebSocket");
// Subscribe to BTC/USDT quotes
ws.send(JSON.stringify({
event: "subscribe",
channel: "market",
market: "BTC_USDT"
}));
});
ws.on("message", (msg) => {
console.log("Receive message:", msg.toString());
});
import websocket
import json
def on_open(ws):
print("Connected to Pionex WebSocket")
sub_msg = {
"event": "subscribe",
"channel": "market",
"market": "BTC_USDT"
}
ws.send(json.dumps(sub_msg))
def on_message(ws, message):
print("Receive message:", message)
ws = websocket.WebSocketApp(
"wss://ws.pionex.com/ws",
on_open=on_open,
on_message=on_message
)
ws.run_forever()
availableGET /api/v1/common/symbolsTo obtain all supported trading pairs and detailed attributes of Pionex, such as minimum order size, price accuracy, transaction type (spot or contract), etc.
GET https://api.pionex.com/api/v1/common/symbols
{
"code": 0,
"data": [
{
"symbol": "BTC_USDT",
"quoteCurrency": "USDT",
"baseCurrency": "BTC",
"minQty": "0.0001",
"minNotional": "5",
"pricePrecision": 2,
"quantityPrecision": 6,
"tradeEnable": true
},
{
"symbol": "ETH_USDT",
"quoteCurrency": "USDT",
"baseCurrency": "ETH",
"minQty": "0.001",
"pricePrecision": 2,
"quantityPrecision": 6,
"tradeEnable": true
}
]
}
import requests
BASE_URL = "https://api.pionex.com"
def get_symbols():
url = f"{BASE_URL}/api/v1/common/symbols"
res = requests.get(url)
data = res.json()
if data.get("code") == 0:
symbols = data.get("data", [])
print(f"A total of {len(symbols)} trading pairs obtained")
for s in symbols[:10]: # Only display the first 10
print(f"{s['symbol']} ({s['baseCurrency']}/{s['quoteCurrency']})")
else:
print("Acquisition failed:", data)
if __name__ == "__main__":
get_symbols()
To remove only trading pairs that support perpetual contracts, you can use the following simple filtering logic:
perp_symbols = [s for s in data["data"] if ".PERP" in s["symbol"]]
/api/v1/market/tickersGet instant prices.The following program will callhttps://api.pionex.com/api/v1/common/symbols, automatically print out the structure (key and data type) of the returned JSON, making it easier to understand the actual format.
import requests
import json
def print_json_structure(data, indent=0):
"""Print JSON structure recursively"""
space = " " * indent
if isinstance(data, dict):
for k, v in data.items():
if isinstance(v, (dict, list)):
print(f"{space}{k}: {type(v).__name__}")
print_json_structure(v, indent + 1)
else:
print(f"{space}{k}: {type(v).__name__}")
elif isinstance(data, list) and data:
print(f"{space}[list] item type: {type(data[0]).__name__}")
print_json_structure(data[0], indent + 1)
def get_pionex_symbols_format():
url = "https://api.pionex.com/api/v1/common/symbols"
res = requests.get(url)
res.raise_for_status()
data = res.json()
print("Root level structure:")
print_json_structure(data)
if __name__ == "__main__":
get_pionex_symbols_format()
Root level structure:
code: int
data: list
[list] item type: dict
symbol: str
baseCurrency: str
quoteCurrency: str
pricePrecision: int
quantityPrecision: int
minQty:str
minNotional: str
tradeEnable: bool
json.dumps(data, indent=2)Full content available.Obtain the K-line (Candlestick / OHLCV) data of the specified trading pair, sourced from Pionex public market data.
| parameter | Type | Is it necessary | illustrate |
|---|---|---|---|
| symbol | string | yes | Trading pair (example: BTC_USDT or BTC_USDT.PERP) |
| interval | string | yes | Time interval, such as 1M, 5M, 15M, 30M, 60M, 4H, 8H, 12H, 1D |
| endTime | number (milliseconds) | no | end time (millisecond timestamp) |
| limit | number | no | Number of data obtained, default 100, range 1-500 |
{
"result": true,
"data": {
"klines": [
{
"time": 1691649240000,
"open": "1851.27",
"close": "1851.32",
"high": "1851.32",
"low": "1851.27",
"volume": "0.542"
}
]
},
"timestamp": 1691649271544
}
import requests
BASE_URL = "https://api.pionex.com"
def get_klines(symbol: str, interval: str, end_time: int = None, limit: int = 100):
params = {
"symbol": symbol,
"interval": interval,
"limit": limit
}
if end_time is not None:
params["endTime"] = end_time
response = requests.get(f"{BASE_URL}/api/v1/market/klines", params=params)
result = response.json()
if result.get("result") and "data" in result:
return result["data"]["klines"]
else:
raise Exception(f"Failed to obtain K line: {result}")
if __name__ == "__main__":
# Example: Obtain the latest 50 15-minute K-lines of the BTC_USDT perpetual contract
symbol = "BTC_USDT.PERP"
interval = "15M"
klines = get_klines(symbol, interval, limit=50)
for k in klines:
print(k)
Pionex does not have a single"Get support for grid trading pairs"dedicated API, but can be accessed viaGET /api/v1/market/tickersGet all trading pairs and filter them out.PERPType (USDT Perpetual Contract) to get the list of trading pairs supported by Futures Grid.
import requests
BASE_URL = "https://api.pionex.com"
def get_perp_pairs():
url = f"{BASE_URL}/api/v1/market/tickers"
res = requests.get(url)
data = res.json()
perp_pairs = []
if "data" in data:
for item in data["data"]:
market = item.get("symbol", "")
# Perpetual contract trading pairs usually end with .PERP
if ".PERP" in market:
perp_pairs.append(market)
return perp_pairs
if __name__ == "__main__":
pairs = get_perp_pairs()
print("Supported perpetual contract grid trading pairs:")
for p in pairs:
print(p)
Supported perpetual contract grid trading pairs:
BTC_USDT.PERP
ETH_USDT.PERP
SOL_USDT.PERP
LINK_USDT.PERP
...
.PERP) can be used directly with Futures Grid Bot.The Max Coin API is a set of application programming interfaces provided by Max Exchange that allows developers to programmatically access its cryptocurrency trading functionality. Developers can automate transactions, retrieve market data, and manage assets through APIs.
The following is an example of obtaining market data through the API:
GET https://max-api.maicoin.com/api/v1/ticker?market=btctwd
This request will return real-time market data for BTC/TWD, including price, volume, etc.
The following is a simple example of calling Max API using Python language:
import requests
BASE_URL = "https://max-api.maicoin.com"
def get_ticker(pair):
endpoint = "/api/v1/ticker"
params = {"market": pair}
response = requests.get(BASE_URL + endpoint, params=params)
return response.json()
# Get BTC/TWD market data
ticker_data = get_ticker("btctwd")
print(ticker_data)
Bitcoin usageProof of Work (PoW)As a consensus mechanism to achieve a decentralized accounting system. The core goal of PoW is to allow nodes to compete to solve mathematical problems to determine who has the accounting rights, ensuring that data cannot be tampered with at will.
Miners will prioritize transactions based on the following criteria to increase the chance of successfully packaging blocks and earning fees:
When the transaction volume is too large, congestion and rising handling fees will occur. To improve this problem, various extension solutions are proposed:
| Field name | Size (bytes) | illustrate |
|---|---|---|
| block size | 4 | The total size of the entire block (including header and all transaction data) (in bytes) |
| block header | 80 | The header information of the block is used for verification and linking the previous and later blocks. |
| transaction counter | 1 ~ 9 | The number of transactions, expressed as a variable-length integer (VarInt), indicating how many transactions there are in this block |
| transactions | variable | All actual transaction data, each transaction includes inputs and outputs |
The length is fixed at 80 bytes and the content is as follows:
| Field name | data type | length | illustrate |
|---|---|---|---|
| version | int32 | 4 | Block version, representing acceptable block verification rules |
| previous block hash | char[32] | 32 | The hash value of the previous block |
| merkle root | char[32] | 32 | Merkle Tree root of all transaction hashes |
| timestamp | uint32 | 4 | Block creation time (UNIX timestamp) |
| bits | uint32 | 4 | A compressed representation of the target difficulty |
| nonce | uint32 | 4 | In order to find the changing value of the valid block hash |
Use VarInt (variable-length integer) format to indicate how many transactions are in the block:
Each transaction data contains the following main parts (lengths vary):
The first transaction is usuallycoinbase trading, which is a special transaction for miners to receive block rewards, and does not include input.
import time
class Transaction:
def __init__(self, sender, receiver, amount, fee):
self.sender = sender
self.receiver = receiver
self.amount = amount
self.fee = fee
self.timestamp = time.time()
def __repr__(self):
return f"[Tx: {self.sender} → {self.receiver}, ${self.amount}, fee: {self.fee}]"
class Node:
def __init__(self, name):
self.name = name
self.peers = []
self.transaction_pool = []
def connect(self, peer):
if peer not in self.peers:
self.peers.append(peer)
peer.connect(self) # Two-way connection
def receive_transaction(self, tx):
if tx not in self.transaction_pool:
self.transaction_pool.append(tx)
print(f"{self.name} received transaction: {tx}")
self.broadcast(tx)
def broadcast(self, tx):
for peer in self.peers:
peer.receive_transaction(tx)
classMiner(Node):
def mine_block(self):
print(f"\n⛏️ {self.name} starts packing block")
# Sort by high handling fee, take up to 5 transactions
sorted_txs = sorted(self.transaction_pool, key=lambda tx: tx.fee, reverse=True)
selected = sorted_txs[:5]
print(f"{self.name} packaged transaction:")
for tx in selected:
print(f" - {tx}")
#Clear processed transactions
self.transaction_pool = [tx for tx in self.transaction_pool if tx not in selected]
# Create nodes and miners
A = Node("Node A")
B = Node("Node B")
C = Miner("Miner C")
# Connect node network
A.connect(B)
B.connect(C)
# User issues transaction
tx_list = [
Transaction("Alice", "Bob", 2.0, 0.0005),
Transaction("Eve", "Tom", 1.2, 0.0009),
Transaction("Joe", "Mary", 3.5, 0.0002),
Transaction("Rick", "Sam", 0.8, 0.0015),
Transaction("Ann", "Lily", 1.7, 0.0001)
]
for tx in tx_list:
print(f"\nUser sends transaction: {tx}")
A.receive_transaction(tx)
time.sleep(0.2)
# Miners start packaging
C.mine_block()
The Ethereum Virtual Machine (EVM for short) is the core component of Ethereum and is responsible for executing smart contracts. The EVM provides a sandbox environment that allows developers to run code on it without worrying about affecting other parts of the Ethereum network.
Smart contracts are self-executing, immutable contracts whose execution is handled by the EVM. Developers typically write smart contracts using high-level programming languages such as Solidity, which are then compiled into bytecode that the EVM understands.
Solidity is the most commonly used programming language on Ethereum, with a syntax similar to JavaScript. Here is a simple Solidity smart contract example:
pragma solidity ^0.8.0;
contract SimpleStorage {
uint public storedData;
function set(uint x) public {
storedData = x;
}
function get() public view returns (uint) {
return storedData;
}
}
The above contract contains a variable used to store integer datastoredData, and functions to set and get that data.
When a user executes a smart contract on the Ethereum network, the following steps occur:
The computing resources of EVM are limited. In order to prevent network abuse, EVM usesGasMechanism to calculate and charge transaction fees. Each operation has its corresponding Gas cost, and users need to provide sufficient Gas when submitting transactions.
To pay for the computing resources required to execute smart contracts.
The EVM is the core of the Ethereum network and provides a powerful environment to run smart contracts. By using programming languages such as Solidity, developers can create a variety of decentralized applications (dApps) and leverage the capabilities of the EVM to implement complex logical operations and transaction processing.
Node.jsandnpmHardhat:npm install --save-dev hardhatnpx hardhat, select "Create a basic sample project"existcontractsAdded to folderLendingProtocol.soland paste your Solidity contract.
AtscriptsFolder creationdeploy.js, the content is as follows:
async function main() {
const [deployer] = await ethers.getSigners();
console.log("Deployment account:", deployer.address);
const TokenAddress = "0xYourTokenAddressHere";
const LendingProtocol = await ethers.getContractFactory("LendingProtocol");
const lending = await LendingProtocol.deploy(TokenAddress);
await lending.deployed();
console.log("LendingProtocol deployed successfully:", lending.address);
}
main().catch((error) => {
console.error(error);
process.exitCode = 1;
});
Start the Hardhat test chain:
npx hardhat node
Open another terminal to deploy:
npx hardhat run scripts/deploy.js --network localhost
hardhat.config.jsAdd in:require("@nomiclabs/hardhat-ethers");
module.exports = {
networks: {
goerli: {
url: "https://goerli.infura.io/v3/your API key",
accounts: ["0xyour private key"]
}
},
solidity: "0.8.20"
};
Then deploy:
npx hardhat run scripts/deploy.js --network goerli
After deployment, the contract address will be output, which can be used for front-end integration and interaction.
This lending agreement is a smart contract running on the Ethereum Virtual Machine (EVM). Users can deposit assets to earn interest or lend assets to pay interest through this contract. This protocol supports ERC-20 tokens and has core functions such as lending, saving, and clearing.
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;
interface IERC20 {
function transferFrom(address sender, address recipient, uint amount) external returns (bool);
function transfer(address recipient, uint amount) external returns (bool);
function balanceOf(address account) external view returns (uint);
function approve(address spender, uint amount) external returns (bool);
}
contract LendingProtocol {
IERC20 public token;
address public owner;
uint public interestRate = 5; // annual interest rate 5%
mapping(address => uint) public deposits;
mapping(address => uint) public borrows;
constructor(address _token) {
token = IERC20(_token);
owner = msg.sender;
}
function deposit(uint amount) external {
require(amount > 0, "Amount must be greater than 0");
token.transferFrom(msg.sender, address(this), amount);
deposits[msg.sender] += amount;
}
function borrow(uint amount) external {
require(amount > 0, "Amount must be greater than 0");
uint collateral = deposits[msg.sender];
require(collateral >= amount * 2, "Insufficient collateral");
borrows[msg.sender] += amount;
token.transfer(msg.sender, amount);
}
function repay(uint amount) external {
require(amount > 0, "Amount must be greater than 0");
require(borrows[msg.sender] >= amount, "Insufficient borrowing");
borrows[msg.sender] -= amount;
token.transferFrom(msg.sender, address(this), amount);
}
function withdraw(uint amount) external {
require(deposits[msg.sender] >= amount, "Insufficient balance");
require(borrows[msg.sender] == 0, "There are unpaid loans");
deposits[msg.sender] -= amount;
token.transfer(msg.sender, amount);
}
}
BSC (Binance Smart Chain) is a public chain compatible with EVM (Ethereum Virtual Machine), so Solidity smart contracts written on Ethereum can be almost directly deployed to BSC. The main difference is that the network connection and gas fees during deployment are priced in BNB.
// SPDX-License-Identifier: MIT
pragma solidity ^0.8.0;
contract SimpleStorage {
uint private value;
function setValue(uint _value) public {
value = _value;
}
function getValue() public view returns (uint) {
return value;
}
}
Deploy using Hardhat, inhardhat.config.jsConfigure the BSC network in:
require("@nomiclabs/hardhat-ethers");
module.exports = {
solidity: "0.8.20",
networks: {
bscTestnet: {
url: "https://data-seed-prebsc-1-s1.binance.org:8545/",
chainId: 97,
gasPrice: 20000000000,
accounts: ["0xyour private key"]
},
bscMainnet: {
url: "https://bsc-dataseed.binance.org/",
chainId: 56,
gasPrice: 20000000000,
accounts: ["0xyour private key"]
}
}
};
async function main() {
const [deployer] = await ethers.getSigners();
console.log("Deployment account:", deployer.address);
const SimpleStorage = await ethers.getContractFactory("SimpleStorage");
const storage = await SimpleStorage.deploy();
await storage.deployed();
console.log("Contract deployed:", storage.address);
}
main().catch((error) => {
console.error(error);
process.exitCode = 1;
});
npm install --save-dev hardhat @nomiclabs/hardhat-ethers ethersnpx hardhat run scripts/deploy.js --network bscTestnetUse Python and a random forest model to predict the probability of a stock rising or falling. This example uses stock data from Yahoo Finance and uses technical indicators to train the model, and finally outputs the probability of the stock rising and falling.
First, we need to install and import some Python packages:
pip install yfinance scikit-learn pandas numpy
Then import these necessary libraries:
import pandas as pd
import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score
import yfinance as yf
Next, we download the stock's historical data from Yahoo Finance and calculate the Simple Moving Average (SMA) as a technical indicator.
# Download Apple stock data from Yahoo Finance
symbol = 'AAPL'
data = yf.download(symbol, start='2020-01-01', end='2023-01-01')
# Calculate 10-day and 50-day simple moving averages (SMA)
data['SMA_10'] = data['Close'].rolling(window=10).mean()
data['SMA_50'] = data['Close'].rolling(window=50).mean()
#Set an increase or decrease target. If the closing price of the next day is higher than the closing price of the day, it will be 1 (indicating an increase), otherwise it will be 0 (indicating a decrease)
data['Target'] = np.where(data['Close'].shift(-1) > data['Close'], 1, 0)
# Remove missing values
data.dropna(inplace=True)
We will use moving averages (SMA) and closing prices as features and split the data into training and test sets.
# Select features
features = ['SMA_10', 'SMA_50', 'Close']
X = data[features]
y = data['Target']
# Split the training set and test set (80% training, 20% testing)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, shuffle=False)
We use a random forest model to train the data and predict the rise and fall probabilities of stocks in the test set.
#Initialize the random forest classifier
model = RandomForestClassifier(n_estimators=100, random_state=42)
#Train model
model.fit(X_train, y_train)
# Predict the rise and fall in the test set
y_pred = model.predict(X_test)
y_prob = model.predict_proba(X_test) # Get the probability of rise and fall
We can calculate the accuracy of the model and display the probability of rise and fall for each day.
# Calculate model accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Model accuracy: {accuracy:.2f}")
# Display the rise and fall probabilities of the first 5 days in the test set
for i in range(5):
print(f"Day {i+1}: rising probability={y_prob[i][1]:.2f}, falling probability={y_prob[i][0]:.2f}")
Based on the probability of an increase, you can set a threshold to decide whether to make a buy operation. For example, if the probability of an increase exceeds 70%, buy.
# Set the threshold for rising probability
threshold=0.7
# Make buying decisions based on probability
for i in range(len(y_prob)):
if y_prob[i][1] > threshold:
print(f"Recommended to buy on day {i+1}, predicted probability of increase={y_prob[i][1]:.2f}")
else:
print(f"It is not recommended to buy on day {i+1}, predicted probability of increase={y_prob[i][1]:.2f}")
This example shows how to use a random forest model to predict the probability of a stock's rise or fall and make trading decisions based on the prediction results. This is a simple but effective way to improve the accuracy of your trading decisions.
The public subscription announcement page of the Taiwan Stock Exchange can be accessed at the following URL:
Public Subscription Announcement
Using Python withrequestsandBeautifulSoup, you can grab the public subscription information on the page.
import requests
from bs4 import BeautifulSoup
import pandas as pd
# Capture the public subscription announcement page
url = "https://www.twse.com.tw/zh/announcement/public.html"
headers = {"User-Agent": "Mozilla/5.0"}
response = requests.get(url, headers=headers)
if response.status_code == 200:
soup = BeautifulSoup(response.text, 'html.parser')
tables = pd.read_html(response.text)
if tables:
df = tables[0] # Get the first table
print(df)
else:
print("Table data not found")
else:
print("Unable to connect to the public subscription announcement page")
SeleniumSimulate browser operations.CurrencyAPI provides real-time exchange rate information. Here is sample code:
import requests
url = 'https://api.currencyapi.com/v3/latest'
params = {
'apikey': 'Your API key',
'base_currency': 'USD',
'currencies': 'TWD'
}
response = requests.get(url, params=params)
data = response.json()
usd_to_twd = data['data']['TWD']['value']
print(f"1 US dollar equals {usd_to_twd} New Taiwan dollars")
Notice:You need to register at CurrencyAPI and obtain an API key before you can use it.
ExchangeRatesAPI also provides real-time exchange rate query services:
import requests
url = 'https://api.exchangeratesapi.io/latest'
params = {
'access_key': 'Your API key',
'base': 'USD',
'symbols': 'TWD'
}
response = requests.get(url, params=params)
data = response.json()
usd_to_twd = data['rates']['TWD']
print(f"1 US dollar equals {usd_to_twd} New Taiwan dollars")
Please first register at ExchangeRatesAPI to obtain an API key and replace 'your API key' in the code.
If you don't want to call the API directly, you can use a third-party Python packageforex-python:
from forex_python.converter import CurrencyRates
cr = CurrencyRates()
usd_to_twd = cr.get_rate('USD', 'TWD')
print(f"1 US dollar equals {usd_to_twd} New Taiwan dollars")
Install package command:
pip install forex-python
If you do not use the API, you can use web crawler technology to retrieve the real-time exchange rate of the US dollar against the Taiwan dollar directly from the Currency.Wiki website.
The following Python packages need to be installed:
pip install requests pip install beautifulsoup4
import requests
from bs4 import BeautifulSoup
# Set destination URL
url = "https://currency.wiki/usd_twd"
#Send a GET request to obtain web page content
response = requests.get(url)
soup = BeautifulSoup(response.text, 'html.parser')
# Find specific tags and categories
span_tag = soup.find('span', class_='unit_secondary_value')
rate = span_tag.text
print(f"1 US dollar equals {rate} Taiwan dollars")
You can directly obtain real-time exchange rate information through Python crawler technology, but you need to pay attention to web page structure changes and compliance issues. It is suitable for small-scale applications or learning purposes.
email: [email protected]