标签：#python

We have tools for checking whether a query is injectable. We have linters, scanners, ORMs, parameterized queries, and database policies. But after the database returns rows, most applications simply trust that the result set matches the operation that asked for it. queryguard starts there. The query may be safe. The result may still be wrong. SQL injection taught us to distrust query construction. Parameterized queries answered the question: Did the user control the query structure? That question is well understood. The tooling is mature. But it is a different question from the one queryguard asks: Did this operation receive only the rows and fields it was allowed to receive? Those two questions are not the same. A perfectly safe parameterized query can still return the wrong row — because a predicate was dropped, a join widened the result, a developer selected a column they shouldn't have, or a query was rewritten without updating its scope contract. queryguard is not a database firewall. It is not a SQL injection scanner. It is not an ORM plugin. It is a contract check for observed result sets. Where it sits The hook position is the core design decision. queryguard sits immediately after cursor execution — before any result shaping, filtering, serialization, or response mapping. cursor = conn . execute ( sql , bindings ) rows = [ dict ( row ) for row in cursor . fetchall ()] evidence = queryguard . run_check ( contract , { " contract_id " : " user_profile_lookup " , " contract_version " : " 0.1.0 " , " params " : { " user_id " : user_id }, " session " : { " tenant_id " : tenant_id }, " result " : rows , }) if evidence [ " verdict " ] != " PASS " : raise QueryguardViolation ( evidence ) return rows Not at the HTTP layer. Not inside the ORM. Not at the API gateway. Immediately after the cursor returns rows — while the result is still raw, before anything shapes or discards it. This is intentional. If rows are shaped before queryguard sees them, queryguard cannot det

GitHub热门项目 | A list of free LLM inference resources accessible via API. | Stars: 24,157 | 100 stars today | 语言: Python

GitHub热门项目 | 小红书笔记 | 评论爬虫、抖音视频 | 评论爬虫、快手视频 | 评论爬虫、B 站视频 ｜ 评论爬虫、微博帖子 ｜ 评论爬虫、百度贴吧帖子 ｜ 百度贴吧评论回复爬虫 | 知乎问答文章｜评论爬虫 | Stars: 52,587 | 347 stars today | 语言: Python

GitHub热门项目 | Transforms complex documents like PDFs and Office docs into LLM-ready markdown/JSON for your Agentic workflows. | Stars: 69,150 | 524 stars today | 语言: Python

GitHub热门项目 | AI 时代的伯克希尔：基于 Claude Code 的价值投资研究框架。巴菲特·芒格·段永平·李录四大师方法论 + 多Agent并行研究。| AI-era Berkshire: a value investing research framework built on Claude Code. 4 masters' methodologies + multi-agent adversarial analysis. | Stars: 1,563 | 201 stars today | 语言: Python

AI harnesses are the new vendor lock-in. To swap across harnesses easily without rewriting your app, LiteLLM launched the Lite-Harness SDK . Run your prompt across different harnesses: from lite_harness import query , AgentOptions prompt = " Fix the failing test " # Claude Code harness async for message in query ( prompt = prompt , options = AgentOptions ( harness = " claude-code " , model = " claude-opus-4-8 " ), ): print ( message ) # Codex harness async for message in query ( prompt = prompt , options = AgentOptions ( harness = " codex " , model = " gpt-5.5 " ), ): print ( message ) To enable cost controls, fallbacks, and logging, point it to your LiteLLM AI Gateway: export LITELLM_API_BASE = https://litellm.your-company.com/v1 export LITELLM_API_KEY = sk-litellm-... Engineer's Takeaway: This SDK unifies how you invoke the agents, not how they run internally. Each harness keeps its native loop and tool-calling semantics. It is perfect for A/B testing agent performance and centralizing costs, but remember it is in public beta, so custom tool injection might require extra work! The Problem I Had My team was building an internal bot to fix failing CI/CD tests. We had three engineers advocating for three different harnesses: one wanted Claude Code, another Codex, and another Pi AI. Without an abstraction layer, we would have had to maintain three forks of the same bot , with three different SDKs, three logging systems, and three ways to track costs. It would have been an impossible maintenance burden. How Lite-Harness Helped The SDK solved that exact pain point in three concrete dimensions : 1. Unified Invocation (Time Savings) Instead of maintaining three separate implementations, I had a single query() that routed to whichever harness I wanted. Switching from Claude Code to Codex was literally just changing a string in the options. This allowed us to do real A/B testing in production for two weeks without rewriting any core logic. 2. Cost Observability (The Killer

Building a unified market intelligence platform for traders, analysts, researchers, and developers. After months of development, Jungletrade is now publicly available. The idea behind Jungletrade is simple: modern market analysis has become fragmented. Market data, indicators, analytical models, and trading signals are often distributed across multiple platforms, forcing users to maintain several subscriptions, workflows, and dashboards just to build a complete market view. We wanted to explore a different approach. 📊 The Problem Most market platforms focus on a specific layer of the analytical stack: Raw data Technical indicators Quantitative models Trading signals Each layer provides value, but users are frequently required to move between multiple tools to connect the pieces. Our goal was to create a modular ecosystem where these layers can coexist within a single platform. 🧭 The Jungletrade Ecosystem Today, JungleTrade provides four product categories: 📦 Data Structured datasets for market research and discovery. 🧠 Models Analytical frameworks designed to identify patterns and relationships within market data. 📈 Indicators Tools that transform raw information into actionable insights. ⚡ Triggers Event-driven signals designed to highlight potential market opportunities. 🔍 Built for Transparency One design decision was particularly important to us: every product should explain itself. Each product includes: Product description Key features Use cases Interpretation guidelines Methodology overview The objective is not simply to provide charts but to explain the problem being solved and how the underlying analysis works. 🔌 API First All products available through the platform are also accessible through API endpoints. Developers interested in integrating JungleTrade data into their own applications, dashboards, or research pipelines can request a demo API key through the platform. 🏗️ Architecture JungleTrade is built using a modular, service-oriented architecture des

I started as a Java developer, but for some time now, I have broadened my horizons. Recently, I thought about how early languages were dedicated to a single target and platform, and now they are broadening their focus. In this post, I want to write down my thoughts in the hope that it may be useful to others, probably to my future self. Definitions You may have been wondering about the title terms. I'm pretty sure that if you read this post, you have a pretty good picture of what a programming language is. Some may disagree on some finer points or raise a hair-splitting one, but it's not a PhD thesis, only a post on my blog. I must define what I mean by target and platform in the context of this post before going further. Target A target only makes sense in the context of compiled programming languages. For example, C's target is native code , and Java's is bytecode . Platform A platform is the system that will ultimately run the target. Native code runs on the operating system; bytecode on the JVM. Early programming languages Early programming languages had a single target and platform. I mentioned C and Java, but Ruby, Python, JavaScript, etc., were all the same. Programming language Target Platform C Native code Operating system C++ Native code Operating system Java Bytecode JVM Python - Python runtime TypeScript JavaScript Browser & server-side JS JavaScript - Browser I believe it was the case for a long time. It changed at some point, though. Multi-target is the new black The first time I heard about multi-target was in Scala. Scala came from the era of single-target and targeted bytecode on the JVM platform. However, in 2015, Martin Odersky announced Scala.js, which added JavaScript to Scala's target. The original article was published on InfoWorld, but it seems to have redirection issues nowadays. Here's the introduction on a copy: Scala, developed as a functional and object-oriented language for the JVM, is now multiplatform, with developers using it in abun

Building a real-time crypto trading bot sounds like a weekend project — until exchange APIs return cryptic errors, WebSocket connections drop mid-trade, and rate limits turn your strategy into a debugging nightmare. After building my own bot from scratch, I learned that reliability is what separates a hobby script from a system that actually survives in production. This guide walks through the entire process: modular bot architecture, a real-time trading loop, plug-in strategies, backtesting, paper trading, and deployment to a $5 VPS — with production reliability patterns baked in from day one. Full source code included — the free AlgoTrak Backtest Lab on GitHub has 5 classic strategies, a complete backtesting engine, and Jupyter notebooks to get started immediately. Architecture Overview Before writing code, here's the modular structure we'll build: crypto_bot/ ├── strategies/ │ ├── rsi_strategy.py │ ├── macd_strategy.py │ └── ... # Plug in your own ├── core/ │ ├── trader.py # Data fetching + order execution │ └── logger.py # File + DB logging ├── config/ │ └── settings.json ├── cli.py # Entry point ├── bot.py # Main loop └── logs/ Each strategy is a standalone Python class. The trader handles exchange communication. The CLI lets you switch between strategies, symbols, and modes (paper vs live) without touching code. Real-Time Trading Loop Here's the core loop that runs every candle interval: while True : df = fetch_ohlcv ( symbol , interval ) signal = strategy . evaluate ( df ) if signal == " BUY " : trader . buy ( symbol , quantity ) elif signal == " SELL " : trader . sell ( symbol , quantity ) sleep ( next_candle_time ()) Three key points: fetch_ohlcv() pulls the latest OHLCV candle data from the exchange Your strategy evaluates the last N candles and returns a signal Orders execute only on valid signals — no guesswork Modular Strategy Example (RSI) Strategies follow a simple class interface. Here's a complete RSI strategy: import pandas as pd import pandas_ta a

Apache Iceberg looked like the answer to everything when we first adopted it. Open format, ACID transactions, time travel, schema evolution. We migrated our Hive tables, ran a few queries, and felt good about life. Three months later, our S3 costs doubled. Queries that used to take 10 seconds were taking 4 minutes. Metadata operations were timing out. Nobody on the team could explain why. That was the beginning of a real education in how Iceberg actually behaves in production. This post covers what I wish someone had told us before we went all-in. The Small Files Problem Is Not Optional Iceberg is append-friendly by design. Every micro-batch write, every streaming insert, every incremental load creates new Parquet files. Each file also gets its own metadata entry. After a week of hourly loads, you might have 10,000 files in a single partition where you wanted 20. The result: Iceberg's metadata layer has to plan queries across thousands of file manifests. Planning takes longer than execution. Your 10-second query becomes a 4-minute query, and your users start filing tickets. Fix: automate compaction from day one. In Spark, compaction is called rewrite_data_files . The basic call looks like this: -- Run this on a schedule, not on-demand CALL iceberg_catalog . system . rewrite_data_files ( table => 'analytics.events' , strategy => 'binpack' , options => map ( 'target-file-size-bytes' , '134217728' , -- 128MB target per file 'min-input-files' , '5' -- only compact if 5+ small files exist ) ) Target file size of 128MB to 512MB is the practical sweet spot. Smaller than that, you still have too many files. Larger, and your query engines cannot parallelize reads efficiently. If you are not using Spark, PyIceberg exposes compaction through the table maintenance API (as of 0.7.x). For Flink or Trino-only shops, schedule compaction as a separate Spark job. Yes, it is annoying, but it is the right call. Hidden Partitioning Is the Feature You Are Probably Ignoring Old Hive parti

Introduction Most developers today are learning how to “use AI APIs.” But that’s not enough anymore. The real shift happening in software engineering is this: We are moving from building APIs → to building AI-powered systems. And that requires a completely different mindset. The Problem with Most AI Tutorials Most tutorials show this: Call OpenAI API Get response Print output That’s it. But in production systems, this approach fails because it ignores: Context management State handling Reliability Tool integration System design In real applications, AI is not a function call — it is an orchestrated system. What an AI System Actually Looks Like A production AI system usually includes: 1. Input Layer Validation Preprocessing Safety checks 2. Reasoning Layer (LLM) Prompt engineering Context injection Model selection 3. Tool Layer APIs Databases Search engines Internal services 4. Memory Layer Conversation history Vector DB / embeddings User context 5. Output Layer Formatting Validation Response filtering Simple Example: From API Call → AI Agent Thinking Instead of this: response = client . chat . completions . create (...) We design something like this: class AIAgent : def __init__ ( self , llm , tools ): self . llm = llm self . tools = tools def run ( self , user_input : str ): context = self . build_context ( user_input ) response = self . llm . chat . completions . create ( model = " gpt-4o-mini " , messages = context , temperature = 0.2 ) return self . post_process ( response ) Now AI becomes: ✔ structured ✔ extendable ✔ production-ready Key Shift in Thinking Old mindset: “How do I call the model?” New mindset: “How do I design the system around the model?” That’s the difference between: ❌ AI script ✅ AI product system Why Tools Matter More Than Prompts Modern AI systems are not just text generators. They are tool-using systems . Examples: Search APIs (RAG systems) Databases (SQL, NoSQL) External APIs Internal business logic This turns AI from “chatbot” into “agent

Hook: Why Your Next Sports Analytics Project Shouldn't Cost a Fortune Last summer, a college student in Ohio built a machine learning model that predicted NBA player performance with 87% accuracy—without spending a single dollar on data. Meanwhile, a startup in London created a real-time football analytics dashboard that rivaled paid enterprise solutions. The secret? Free sports data APIs. The sports data landscape has transformed dramatically. Where teams once paid six figures for proprietary datasets, developers and data scientists now have access to institutional-quality information at zero cost. Whether you're building a fantasy sports optimizer, analyzing player statistics, or creating predictive models, the barrier to entry has never been lower. But not all free APIs are created equal. Some offer comprehensive historical datasets spanning decades. Others provide real-time updates but limited depth. This guide cuts through the noise and delivers a practical, hands-on review of the best free sports data tools available in 2025. Why Free Sports Data Matters Now More Than Ever The democratization of sports data represents a fundamental shift in the industry. Five years ago, accessing granular sports statistics required partnerships with ESPN, official league APIs, or expensive data brokers. Today's ecosystem has flipped that model. The practical advantages: Lower barriers to entry : Students, hobbyists, and early-stage startups can build sophisticated analytics projects without capital constraints Rapid prototyping : Test hypotheses and validate ideas before investing in premium data services Educational access : Learn data engineering, machine learning, and API integration with real-world sports datasets Competitive alternatives : Many free APIs now compete directly with paid solutions in specific domains The catch? Free doesn't mean unrestricted. Rate limits, update frequencies, and feature sets vary dramatically. Understanding what each tool offers—and its limi

If you've ever tried to pull live options data into a Python script, you've probably hit the same wall I did: the cheapest real-time providers start at $99/mo. Here's how to do it for $20/mo — or free if you stay within 1,000 credits/day. What You'll Need Python 3.8+ requests library ( pip install requests ) An API key from market-option.com (free tier available, no card required) Fetching a Full Options Chain import os import requests API_KEY = os . environ [ " MARKET_OPTIONS_KEY " ] BASE_URL = " https://market-option.com/api/v1 " def get_chain ( ticker : str ) -> list [ dict ]: res = requests . get ( f " { BASE_URL } /options/chain/ { ticker } " , params = { " apiKey " : API_KEY }, ) res . raise_for_status () return res . json ()[ " results " ] contracts = get_chain ( " SPY " ) print ( f " { len ( contracts ) } contracts returned " ) print ( contracts [ 0 ]) Each contract in results looks like this: { "details" : { "contract_type" : "call" , "strike_price" : 530 , "expiration_date" : "2026-01-17" , "ticker" : "O:SPY260117C00530000" }, "last_quote" : { "bid" : 3.45 , "ask" : 3.50 , "midpoint" : 3.475 }, "greeks" : { "delta" : 0.42 , "gamma" : 0.031 , "theta" : -0.18 , "vega" : 0.29 }, "implied_volatility" : 0.182 , "open_interest" : 12418 } Filtering by Expiration and Strike def get_near_the_money ( ticker : str , expiration : str , spot : float , width : float = 0.05 ): """ Return contracts within ±width% of spot price. """ contracts = get_chain ( ticker ) low = spot * ( 1 - width ) high = spot * ( 1 + width ) return [ c for c in contracts if c [ " details " ][ " expiration_date " ] == expiration and low <= c [ " details " ][ " strike_price " ] <= high ] atm = get_near_the_money ( " SPY " , " 2026-01-17 " , spot = 530 ) for c in atm : print ( c [ " details " ][ " strike_price " ], c [ " details " ][ " contract_type " ], c [ " last_quote " ][ " bid " ], c [ " greeks " ][ " delta " ], ) Scanning for High IV Contracts def high_iv_scan ( ticker : str , iv_threshold :

A couple weeks ago I posted about ImageX - a dumb little CLI that lets you edit images without googling "resize image online" for the hundredth time. If you haven't seen it, the tldr is: pip install imagex && imagex , a menu pops up, pick what you want, done. No flags to memorize, no syntax to look up - just arrow keys and enter. Everything runs locally on your machine - no uploads, no server ever sees your images, no browser,no ads. Honestly? Didn't expect much. But: 10 stars on GitHub 1 contributor already dropped a PR (Flip feature - thanks! :) A bunch of feature requests in issues So I kept building. What's new in v0.3.0 Three features, one fix: Grayscale / B&W - luminosity conversion or threshold-based true black & white. Slider for the threshold so you can dial in exactly how much black you want. Invert Colors - instant negative. Handles RGBA without destroying alpha. Flip - mirror horizontally, vertically, or both. (PR from a contributor!) Plus: version check on startup (tells you when to upgrade), and all operations now preserve EXIF/ICC metadata instead of silently dropping it. Yeah, that was a bug. Fixed. The code is still stupid simple NAME = " Invert Colors " DESCRIPTION = " Invert image colors (negative effect) " def run ( file , output_path , args ): img = Image . open ( file ) # ... invert logic ... img . save ( output_path ) return True Links GitHub: github.com/kushal1o1/ImageX PyPI: pip install -U imagex PRs welcome. You could write a feature in the time it took to read this.

GitHub热门项目 | Model Context Protocol (MCP) to enable AI LLMs to trade using MetaTrader platform | Stars: 561 | 8 stars today | 语言: Python

GitHub热门项目 | Multi-harness agentic plugin marketplace for Claude Code, Codex CLI, Cursor, OpenCode, GitHub Copilot, and Gemini CLI | Stars: 37,124 | 43 stars today | 语言: Python

GitHub热门项目 | Python SDK, Proxy Server (AI Gateway) to call 100+ LLM APIs in OpenAI (or native) format, with cost tracking, guardrails, loadbalancing and logging. [Bedrock, Azure, OpenAI, VertexAI, Cohere, Anthropic, Sagemaker, HuggingFace, VLLM, NVIDIA NIM] | Stars: 51,382 | 152 stars today | 语言: Python

Why I Stopped Picking AI Models by Hype and Started Picking by Speed Three months ago I almost lost a $14,000 retainer because my chatbot felt sluggish. The client didn't say "your TTFT is too high." They said "it feels dumb." That's freelancer code for "users are bouncing and I'm about to find someone else." I rebuilt that bot in a weekend using a model I'd never even heard of six weeks earlier, dropped average response time from 1.4 seconds to under 300ms, and the client renewed for another six months. That single pivot paid for my rent. So I went down a rabbit hole. I ran the same speed test on every model I could get my hands on through Global API's unified endpoint. Fifteen models. Same prompt. Same regions. Ten iterations each. I'm writing this up because if you're billing by the hour or running a side hustle on a shoestring, speed isn't a vanity metric — it's a profit metric. Let me show you what I found. The Setup (How I Actually Ran the Tests) I'm not a researcher with a rack of GPUs. I'm a guy with a M2 MacBook, a $19/mo Hetzner box, and a stopwatch in the form of Python's time.perf_counter() . Here's how I kept it honest. Date window: All tests run on May 20, 2026 Regions tested: US East (Ohio) and Asia (Singapore) Prompt used: "Explain recursion in 200 words" — boring on purpose, because boring prompts are where most apps actually live Output length: Roughly 150 tokens per run Iterations: 10 runs per model per region, average recorded Streaming: Yes, SSE throughout Endpoint: Global API at https://global-apis.com/v1 I measured two things: TTFT (time to first token — the lag before the user sees anything move) and sustained tokens per second (how fast the words actually arrive after that). Both matter. TTFT is the "is this thing broken?" feeling. Tokens per second is the "is this thing fast?" feeling. Here's the script I used, stripped down to the essentials: import time import requests from statistics import mean API_KEY = " your-global-api-key " BASE_URL

Building point-in-time correct, production-grade feature pipelines — from raw Kafka events to online feature serving in milliseconds, using Spark Structured Streaming and the Databricks Feature Store. Table of Contents The Feature Engineering Problem Architecture Overview Feature Store Concepts: ERD Environment Setup Streaming Feature Pipeline Point-in-Time Correct Training Dataset Generation Writing Features to the Online Store Serving Features at Inference Time Feature Table Reference References The Feature Engineering Problem Feature engineering is where most ML projects silently fail in production. Not because the model is wrong — but because the features the model sees at training time are different from the features it sees at inference time . This is called training-serving skew , and it's the #1 silent killer of ML systems. Three specific failure modes cause it: Online/offline inconsistency — the batch pipeline that computes training features uses different logic than the real-time service that computes inference features Data leakage — training features accidentally include information from the future (e.g. joining on a label that was created after the event) Feature staleness — a model trained on 30-day rolling averages is served features that are 6 hours stale because the pipeline backfills are slow The Databricks Feature Store — now part of Unity Catalog as Feature Engineering in Unity Catalog — solves all three by: Storing feature computation logic alongside the data (no drift between training and serving) Enforcing point-in-time lookups during training dataset creation Providing a unified API for both batch offline reads and low-latency online reads Architecture Overview Feature Store Concepts: ERD Understanding the data model behind the Feature Store is essential for designing correct pipelines. Here's how the entities relate: The critical relationship: a Model Version is bound to a Training Set , which records exactly which feature tables and which p

A deep dive into how Spark transforms your SQL into a physical execution plan — and how Databricks layers Adaptive Query Execution and the Photon vectorized engine on top to squeeze out maximum performance. Table of Contents Why Query Optimization Matters The Catalyst Optimizer Pipeline Stage 1: Parsing — From SQL to Unresolved Logical Plan Stage 2: Analysis — Binding to the Catalog Stage 3: Logical Optimization — Rule-Based Rewrites Stage 4: Physical Planning — Strategies and Cost Models Adaptive Query Execution (AQE) The Photon Engine Reading Explain Plans Tuning Reference Table References Why Query Optimization Matters A Spark query written by a human and a Spark query executed by the engine are often very different things. The gap between them — the optimization — is what separates a job that runs in 3 minutes from one that runs in 3 hours on identical hardware. Databricks compounds Spark's native Catalyst optimizer with two additional layers: Adaptive Query Execution (AQE) — re-optimizes the query at runtime using actual statistics collected mid-job Photon — a C++ vectorized execution engine that replaces the JVM-based Spark executor for eligible operators Understanding all three lets you write queries that cooperate with the engine rather than fight it. The Catalyst Optimizer Pipeline Catalyst is Spark's rule-based and cost-based query optimizer. Every query — whether written in SQL, DataFrame API, or Dataset API — passes through the same four-stage pipeline before a single byte of data is read. Stage 1: Parsing — From SQL to Unresolved Logical Plan # ── Catalyst Stage 1: Parsing ───────────────────────────────────────────────── # Spark uses ANTLR4 to parse SQL into an Abstract Syntax Tree (AST). # At this point column names are NOT validated — the plan is "unresolved". from pyspark.sql import SparkSession spark = SparkSession . builder . appName ( " catalyst-demo " ). getOrCreate () # Both of these produce identical internal representations df_api = ( spark .