docs: 基于最新 main 更新 website-docs，并补充截图占位与面向用户的开篇 - #53
Draft
lyingbug wants to merge 52 commits into
Draft
Conversation
…/mismatched results If the embedding provider returns an empty slice of vectors (e.g. length 0) due to api mismatch or empty input, BatchEmbed will return length 0. Accessing embedding[i] directly in BatchEmbedWithPool results in an out of bounds panic. Because the panic occurs inside a mu.Lock() block, the mutex is never unlocked, leading to a permanent deadlock of the document processing queue. This fix checks the returned embeddings slice length and returns an error instead of panicking, releasing the mutex properly.
…er tree API Folder uploads used to encode their relative directory inside file_name, which made the document list render the whole path as the title and left no way to browse or query a single folder. The directory now lives in a dedicated folder_path column (with a backfill for existing rows), the document list accepts folder_path / folder_recursive filters, and a new GET /knowledge-bases/:id/knowledge/folders endpoint returns the folder tree with per-folder document counts.
The documents tab gains a left folder column built from the folder_path now stored on each knowledge entry. Selecting a folder filters the existing document list (so search, tags, type, status and date filters all keep working), a breadcrumb above the list shows where you are, and uploads made while browsing a folder land in that folder. A per-row folder chip appears only when the list can span folders.
The upload-path builder, breadcrumb/ancestor derivation, folder lookup and tree flattening were inline in the view and the sidebar component. Moving them into folderTree.ts follows the convention of kbListMerge.ts and wikiStatusRefresh.ts, removes the duplicated webkitRelativePath handling, and makes the folder navigation logic directly testable.
The sidebar listed "all documents" and "root" as two rows sitting next to the top-level folders, but those folders are inside the root, so the three rows contradicted each other and reaching the top level for an upload meant selecting a row that did not look like a parent. There is now a single root row with every folder nested beneath it, and what a row lists is decided by one uniform rule at every level: the direct documents, or the whole subtree when the scope toggle is on. "Documents not in any folder" is the root with the toggle off, which is also why the toggle now defaults to on. The upload destination is no longer implied by the sidebar selection either: the confirmation dialog shows which folder the batch lands in, lets it be switched back to the top level, and lists each file's own sub-directory.
…n the tree A folder-only tree cannot show a node for a document that is not in any folder, so single-file uploads were unreachable from the sidebar: a base with 5 documents and one 3-document folder just left two of them unaccounted for. Rather than add a second pseudo-folder row for them (the ambiguity the previous commit removed), the one existing mechanism is made visible: the scope switch is now a labeled segmented control instead of an icon behind a tooltip, and the root row is named after whatever the current scope makes it list - the whole base, or exactly the unfiled documents. The breadcrumb's leading crumb follows the same rule.
The scope switch is gone. It existed because "select a folder" was ambiguous in a list that could not show folders, and turning that ambiguity into a control just moved the problem onto the user. The list now shows what a folder contains: its sub-folders as entries first, then the documents directly inside it. So the mode follows what the user is already doing instead of a switch - browsing shows one level, and any active filter searches the whole subtree flat, with a breadcrumb note saying so. A document uploaded on its own needs no special row either: it sits at the top level next to the folders, exactly where it is. Folders can also be adjusted now, which they could not before - a mis-filed upload previously had to be deleted and re-uploaded. Documents move via the row menu or the batch bar into an existing or newly typed folder, and a folder can be renamed in place, carrying its subtree. Both are plain folder_path updates: nothing is re-parsed, re-chunked or re-embedded.
…e the folder picker to a popup Two things the folder UI got wrong. The rename state used the empty string to mean "nothing is being renamed", but the empty string is also the root folder's own path, so the root row matched and permanently rendered a stray rename input. The sentinel is now null, and rename is additionally gated on the row being a real folder, so no sentinel value can reach the root again. Moving documents also did not deserve a modal dialog: it is one small, reversible choice. In a row's menu the picker is now another level of the menu that is already open, mirroring the "move to knowledge base" sub-menu right next to it; in the batch bar it hangs off the button. Picking a folder performs the move, since there is nothing else to confirm. The upload destination line was also mangling folder names: truncating from the left with direction: rtl reorders CJK text. It now shows the destination's last segment with the full path in the title, reads as a quiet line instead of a grey banner, and only appears when the destination is not the top level.
…alog - Added folder options to the upload confirmation dialog, allowing users to select from existing folders as upload destinations. - Updated folder tree UI to improve folder browsing experience, displaying sub-folders and documents more intuitively. - Enhanced folder picker functionality to support creating new sub-folders directly from the picker. - Improved localization for folder-related strings across multiple languages. - Refactored related components and tests to accommodate new features and ensure proper functionality. This commit streamlines the user experience for managing folders and uploading documents, making it easier to navigate and organize content.
Document folder tree/move/rename endpoints in docs/api, Swagger, and the Go client; align frontend path normalization with server caps; guard rename no-ops and duplicate folder creation; enforce KB ownership on folder rename.
…nderscore scenarios - Updated SQL queries to properly escape LIKE wildcards in folder path filters. - Added tests for renaming knowledge folder paths, including cases with underscores in paths. - Enhanced existing tests to ensure correct folder listing behavior when using underscores.
Re-add worker pool, queue, and task type labels dropped during locale cleanup, register dynamic key prefixes in the i18n audit, and add tests so check-i18n fails if they are removed again.
Refine embed-failure target filtering so wiki/graph-only KBs and missing KB metadata still reach HybridSearch instead of hard search_failed. Add regression tests for KB failure with concurrent web hits and wiki-only empty recall after embedding errors.
The startup migration probed every workspace/provider pair with First(), so each miss emitted a GORM "record not found" log line and a separate round trip. Load all aliases once and index them in memory instead.
* feat(storage): support AWS S3 default credentials * fix(storage): allow clearing S3 credentials * fix static checks and trim locale diff --------- Co-authored-by: qilifan <4359902+qilifan@users.noreply.github.com>
…nal/storageurl The IM channels rewrite resource:// and provider:// references into loadable HTTP URLs because IM clients cannot attach WeKnora credentials to an image fetch. That logic was private to internal/im, so no other surface could reuse it. Move the reference pattern, HTTP-result guard, tenant-aware storage backend resolver and streaming holdback helpers into internal/storageurl, and add a Rewriter that memoises resolutions (a resource:// handle costs one access-grant row per resolution). internal/im now delegates; behaviour is unchanged. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…URLs API responses reference stored files as opaque resource:// handles, so an integrating app had to make a second authenticated call to /files for every image before it could render anything. Add an opt-in that resolves those references server-side into time-limited HTTP(S) URLs, using the same mechanism the IM channels already rely on: - per request: ?resource_urls=public (default: handle, unchanged) - per deployment: RESOURCE_URL_MODE=public Applied to the chat SSE endpoints (knowledge-chat, agent-chat, continue-stream), message history load, and knowledge-search. Streamed answers buffer a trailing incomplete reference so a handle split across two deltas is still rewritten. References that cannot become an HTTP URL (for example local storage with no APP_EXTERNAL_URL) stay handles, so clients keep the /files fallback. Embed channels are deliberately excluded: their visitors are anonymous. SSE payloads share SearchResult pointers and metadata maps with the stream replay buffer and the message being persisted, so those are rewritten as copies. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
…nd SSE wire Includes an end-to-end ContinueStream test that asserts the full SSE wire output. It caught a leak: the references event carries its results twice, and the copy in Data holds the typed []*SearchResult that CopyData did not traverse, so handles reached a caller that asked for public URLs. CopyData now handles the typed slice, []string and map[string]string forms. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Adds a cross-cutting "文件与图片引用" section to docs/api/README.md covering the parameter, the deployment default, the endpoints it applies to, and the two caveats that matter in practice: it needs APP_EXTERNAL_URL (or a publicly reachable storage backend) to produce a link at all, and the links it produces are time-limited but anonymously readable. Also annotates the affected endpoints for Swagger and refreshes the generated files for those parameters only, leaving unrelated drift in the committed swagger output alone. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
In public resource URL mode, building the payload consumes the chunk into the holdback buffer, so returning between build and write would drop it. Reusing emitStreamEvent also removes the duplicated flush-before-completion logic. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Resolving a resource:// handle into a public URL inserted one capability row every time. Rendering an answer resolves the same image across many chunks and re-reading a message history resolves it again on every call, so a read endpoint behaved like a write-heavy one. The table stores only sha256(token), so an existing grant cannot be reused by looking it up — the plaintext token is unrecoverable. Derive the token instead: HMAC-SHA256(SYSTEM_AES_KEY, resource id + time window). The same resource in the same window derives the same token, so the row can be found by its hash and reused; only the first request in a window inserts. The window is half the TTL, so a reused grant always has at least TTL/2 of life left. This keeps the storage properties intact: the table still holds only hashes and the plaintext cannot be reconstructed from a dump without SYSTEM_AES_KEY, while authorization stays entirely on the row, so expiry and revocation still decide whether a token resolves. Without SYSTEM_AES_KEY, grants stay random and per-request as before. A stable URL per window also lets client and CDN caches hit. Stop deleting revoked grants during cleanup. A revoked row has to survive until its expiry, otherwise a token derived for the same resource and window would re-create it and revive the access that was just revoked. grant.revoked_at has no writer yet, so this changes nothing today, but the derived token makes the tombstone load-bearing. Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Follow-up on resource_urls=public. Review turned up one authorization gap plus a
set of correctness problems in the streaming path.
Authorization:
- Embed channels were documented as excluded, but nothing enforced it. The
delegating handlers hand the visitor's own gin context to the shared
handlers, so ?resource_urls=public — and a deployment-wide
RESOURCE_URL_MODE=public — reached anonymous traffic and produced
shareable, credential-free URLs that bypass the channel-scoped
/embed/:channel_id/files proxy. ensureEmbedSession now pins the request
context to handle mode, which covers every delegated endpoint including
future ones. The downgrade is silent so a client that forwards the
parameter keeps working.
- A knowledge-base-restricted API key is refused with 403. Such a key is
already denied /files because a raw storage path cannot be bound to its
allow-list, so handing it anonymous file URLs would reopen that hole from
the other side. The default handle mode stays available to it.
Streaming:
- Rewriter.ref holds its lock across resolution. Resolver implementations keep
an unsynchronised per-provider cache, so the previous unlocked window was a
real data race for any caller sharing a StreamRewriter between goroutines —
which its own doc comment invited. Serialising also collapses a concurrent
duplicate into one signature.
- Release the holdback buffer on every path that ends a stream while the
client is still connected: a user-requested stop, an error event, and
giving up on the event store. Only completion flushed before, so the tail
generated just before a stop was silently dropped.
- A released tail carries the metadata of the event it was cut from instead of
a bare event_id.
- The maxHeldBytes safety valve aligns to a UTF-8 boundary rather than cutting
a character in half, and an unclosed "![…](" only holds back a
whitespace-free destination under 2048 bytes, so prose that merely mentions
Markdown image syntax no longer stalls the stream behind the buffer.
Also resolves the mode before running a knowledge search instead of after, so an
invalid value no longer costs a full retrieval, and warns once per distinct bad
RESOURCE_URL_MODE value instead of once per process.
Co-authored-by: lyingbug <lyingbug@users.noreply.github.com>
Harden public URL mode: stop logging signed URLs at INFO, copy messages before rewriting history responses, reuse winning grants on insert races, sync the Go client resource_urls query param, and add knowledge-search coverage.
Wiki generation sometimes rendered "Write in ." because async workers read empty locale payloads without falling back to context or defaults. Unify language resolution helpers and apply them at enqueue and consume time so every wiki and related async prompt gets a non-empty language.
Centralize ingestion base splitter defaults in chunker.NormalizeSplitterConfig so the preview endpoint applies the same normalization as upload. Regenerate Swagger for parent-child preview fields and add regression tests for single- level preview, default parent/child sizes, and context headers.
A caller that resends CreateFAQEntry before the first call answers could create several rows for the same standard question. Indexing runs inline and the embedding call retries with exponential backoff, so one create can take far longer than an impatient caller waits; meanwhile the duplicate-question check only looked at ChunkStatusIndexed, so a sibling create still mid-flight was invisible and every retry passed the check. - Serialize concurrent creates of the same question behind a Redis SetNX guard, keyed by tenant + knowledge base + question hash, with an in-process fallback when Redis is unavailable so an outage cannot block FAQ authoring. - Widen the duplicate check to every non-deleted status. This also fixes ChunkStatusDefault entries, which are listed to users but never blocked duplicates. - Bound the indexing step of an interactive create so a degraded embedding service cannot stretch the request out; bulk and background indexing keep the full retry budget. - Report a failed rollback instead of discarding the error, since a leftover stored chunk is hidden from the list yet now matches the duplicate check.
Restructure the page header and footer for clearer metadata, add field-level revision diffs with correct incremental comparison (including v1 initial content), and align footer links with in-body wiki link styling.
Add an MCP tool that creates a knowledge entry from raw Markdown text by
calling the existing backend endpoint
POST /knowledge-bases/{id}/knowledge/manual (handler.CreateManualKnowledge).
This completes the "text" half of Tencent#323, whose "file" half was already
covered by create_knowledge_from_file.
The tool defaults to status="publish" so the created entry is chunked,
embedded and made searchable immediately — matching the API/MCP use case
where there is no UI to publish a draft. Callers may pass status="draft"
to save without indexing.
- New client method WeKnoraClient.create_knowledge_from_text.
- New @mcp.tool() create_knowledge_from_text, resolving kb_id by name or
UUID (consistent with create_session / hybrid_search).
- Add focused unittests verifying endpoint, request body shape, status
default/override, and kb_id resolution wiring.
- Bump the stdio tools-list count assertion in test_mcp_transports.py
from 29 to 30 for the newly registered tool.
List create_knowledge_from_file (previously missing from the README tool list) and the new create_knowledge_from_text in README.md and EXAMPLES.md, and add a CHANGELOG entry under Unreleased.
Bumps the cli-deps group in /cli with 3 updates: [github.com/mattn/go-isatty](https://github.com/mattn/go-isatty), [github.com/mattn/go-runewidth](https://github.com/mattn/go-runewidth) and [github.com/modelcontextprotocol/go-sdk](https://github.com/modelcontextprotocol/go-sdk). Updates `github.com/mattn/go-isatty` from 0.0.22 to 0.0.24 - [Commits](mattn/go-isatty@v0.0.22...v0.0.24) Updates `github.com/mattn/go-runewidth` from 0.0.24 to 0.0.27 - [Commits](mattn/go-runewidth@v0.0.24...v0.0.27) Updates `github.com/modelcontextprotocol/go-sdk` from 1.6.1 to 1.7.0 - [Release notes](https://github.com/modelcontextprotocol/go-sdk/releases) - [Commits](modelcontextprotocol/go-sdk@v1.6.1...v1.7.0) --- updated-dependencies: - dependency-name: github.com/mattn/go-isatty dependency-version: 0.0.24 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cli-deps - dependency-name: github.com/mattn/go-runewidth dependency-version: 0.0.27 dependency-type: direct:production update-type: version-update:semver-patch dependency-group: cli-deps - dependency-name: github.com/modelcontextprotocol/go-sdk dependency-version: 1.7.0 dependency-type: direct:production update-type: version-update:semver-minor dependency-group: cli-deps ... Signed-off-by: dependabot[bot] <support@github.com>
* feat(datasource): add Feishu/Lark Drive (云盘) connector Add a new Drive (云盘) data source alongside the existing Feishu wiki connector, letting users sync documents/files under a user-supplied Drive folder_token. Shares the feishu connector package (Client/Config/Region/ export/download/error-handling) with the wiki connector; only resource enumeration and fetch dispatch differ. Backend: - types/datasource.go: ConnectorTypeFeishuDrive / ConnectorTypeLarkDrive - region.go: RegionFeishuDrive / RegionLarkDrive + driveFolderURL - types.go: driveFile / driveFileListResponse / partialDriveFileListError / feishuDriveCursor - client.go: ListDriveFiles / ListDriveFilesRecursiveFrom (shortcut expansion via shortcut_info, visited cycle guard, root-folder rejection) - drive_connector.go: DriveConnector implementing StreamingConnector (FetchStream) + FetchAll + FetchIncremental fallback paths, mirroring the wiki connector's three cursor semantics (resume fast-path, failure-does-not-advance-cursor, toSyncCursor snapshot), plus ListResources / ResolveResourceAncestors (top-down shared traversal, best-effort) / fetchDriveFileContent / parseDriveResourceID - container.go + connector.go: register two Drive instances + metadata Incremental detection uses modified_time returned directly by the list API (verified) - no batch_query/metas call needed. Shortcuts are expanded at list-parse time via shortcut_info; target_type is never "folder". Frontend: - DataSourceEditorDialog.vue: connectorDefs for feishu_drive/lark_drive, Drive root folder_token input + load button before the lazy-load tree, edit-mode prefill from resource_ids - datasourceIcons.ts: reuse feishu/lark icons - i18n (zh-CN/en-US/ko-KR/ru-RU): connector labels, descriptions, drive input group Co-Authored-By: Claude <noreply@anthropic.com> * fix(datasource): refine Feishu Drive connector UI, channel and root folder A batch of fixes from end-to-end verification of the Feishu/Lark Drive connector, all aimed at matching the wiki connector's behavior: Channel (knowledge "source" label): - Add ChannelFeishuDrive / ChannelLarkDrive constants so Drive docs show "飞书云盘" / "Lark 云盘" instead of "unknown", distinct from the wiki connector's "飞书" (ChannelFeishu). - fetchDriveFileContent / appendDriveFileListFailureItems select the channel by region via a new driveChannel() helper. - ingestItem now prefers metadata["channel"] over ds.Type so the connector-supplied channel value reaches the knowledge base. - doc-content.vue channelLabelMap maps feishu_drive/lark_drive. Frontend Drive picker UX: - loadDriveRoot persists the new folder_token in edit mode too (previously skipped updates in edit mode, so listResources read the old token). - The folder_token input is now always visible alongside the tree (not a switch); the tree area shows a "load a folder first" placeholder until the first successful load. - extractDriveFolderToken accepts a bare token or a Feishu Drive folder URL (https://xxx.feishu.cn/drive/folder/<token>) and extracts the token. - Edit mode auto-loads when a saved folder_token exists, and reveals pre-existing selections via revealExistingSelections. - Classify Drive list errors (403/forbidden, 401/auth, not-found) into actionable i18n hints. Root folder name & selection: - Add GetDriveFolderMeta (GET /drive/explorer/v2/folder/:folderToken/meta) to resolve the root folder's human-readable name; the list API only returns children. Best-effort, falls back to the token. - Fix root folder ExternalID to be the bare rootFolderToken (no ":token" suffix) so it matches the saved resource_id form and the direct children's ParentID, which previously broke selection matching on edit. Co-Authored-By: Claude <noreply@anthropic.com> * fix(datasource): sync a single selected Drive file without folder error When a user selects a single file (not a folder) in the Drive picker, the resourceID is "rootFolderToken:fileToken". FetchStream/FetchAll/ FetchIncremental previously passed fileToken straight to ListDriveFilesRecursiveFrom, which expects a folder token - the Feishu list API returns 1061002 (params error) for a file token, failing the whole sync with "all fetched items failed". The wiki connector resolves a single selected node via GetWikiNode; Drive has no single-file meta API (folder meta only works on folders, and metas/batch_query requires doc_type as input - a chicken-and-egg). Instead, listDriveFilesForResource walks the root folder subtree (the file's parent) and filters to the selected fileToken, yielding the full driveFile (type/name/modified_time) needed for fetchDriveFileContent. A sub-folder selection (fileToken is itself a folder) still walks that sub-folder directly - ListDriveFilesRecursiveFrom accepts a folder token, so no filtering is needed there. Co-Authored-By: Claude <noreply@anthropic.com> * ci: add build-test-images workflow for tar.gz artifact export Manual-only workflow_dispatch that builds linux/amd64 Docker images for the current branch and exports them as .tar.gz artifacts downloadable from the Actions run page. No Docker Hub push, no secrets required. - build-ui: builds frontend dist, builds weknora-ui image, exports tar.gz - build-app: builds weknora-app image (CGO + duckdb), exports tar.gz - Input choice: build both / app only / ui only - Artifacts retained 7 days Co-Authored-By: Claude <noreply@anthropic.com> * ci: add build-test-images workflow for tar.gz artifact export Manual-only workflow_dispatch that builds linux/amd64 Docker images for a selected branch and exports them as .tar.gz artifacts downloadable from the Actions run page. No Docker Hub push, no secrets required. - build-ui: builds frontend dist, builds weknora-ui image, exports tar.gz - build-app: builds weknora-app image (CGO + duckdb), exports tar.gz - Input choice: build both / app only / ui only - Artifacts retained 7 days Placed on the default branch so the workflow appears in the Actions list; trigger it with the branch you want to build (e.g. feat/datasource-feishu-drive). Co-Authored-By: Claude <noreply@anthropic.com> * docs(datasource): add feishu shared-blocks extraction design spec Co-Authored-By: Claude <noreply@anthropic.com> * docs(datasource): add feishu shared-blocks implementation plan Co-Authored-By: Claude <noreply@anthropic.com> * feat(datasource): sync feishu drive docx via blocks API with export fallback - Extract wiki/drive shared helpers from connector.go into shared.go - Make fetchDocxWithBlocks a package-level function taking docxFetchInput - Drive docx now renders via blocks API (Markdown + attachment/image sub-items), falling back to the export API on blocks errors or empty renders, mirroring the wiki connector - fetchDriveFileContent returns item slices and takes multimodalEnabled Co-Authored-By: Claude <noreply@anthropic.com> * fix(frontend): label feishu drive knowledge as 飞书云盘 in list The knowledge list source column (DocumentListView.getSourceInfo) only whitelisted feishu/notion/yuque/etc., so feishu_drive/lark_drive docs fell through to the default "上传" label while the wiki connector's "feishu" showed "飞书". Map feishu_drive → 飞书云盘 and lark_drive → Lark 云盘 (i18n keys already existed), and add feishu_drive to the source filter options in KnowledgeBase.vue. Co-Authored-By: Claude <noreply@anthropic.com> * fix(frontend): polish feishu drive resource-step UI in datasource editor - Replace the two stacked t-alert banners under the folder_token input: shareHint becomes an always-visible form-desc line, rootNotSupportedHint moves into a help-circle tooltip next to the label. - folder_token is now required: red asterisk on the label, inline error status + tips on the input (replacing the global MessagePlugin) when empty on 加载/下一步, cleared on input. - Step 2 选择范围 no longer scrolls as a whole: the token input stays fixed and the resource region (placeholder / loading / empty / tree) fills the remaining drawer height, with the tree scrolling internally. Co-Authored-By: Claude <noreply@anthropic.com> * docs(datasource): add feishu drive datasource integration guide User-facing setup guide for feishu_drive/lark_drive: app creation, required scopes (drive:drive:readonly / drive:export:readonly / docx:document:readonly), sharing the folder to the app's group, the four-step wizard, supported file types, sync semantics and FAQ. Also clarify in extractDriveFolderToken's comment that pasted folder URLs are matched by path and work for Lark links too. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(datasource): unify feishu wiki/drive sync into a single generic streaming engine Extract a single generic streaming sync engine (engine.go) shared by the wiki Connector and the Drive DriveConnector. Per-connector differences (node type, listing API, edit-time field, cursor wire format, fetch dispatch, log tag) are isolated behind the NodeOps[N] adapter; FetchAll/FetchIncremental become thin shells over the same engine. Deliberate behaviour changes (design §2.4): the Tencent#2136 failure-doesn't-advance-cursor fix now applies to FetchIncremental too (previously it advanced the cursor before fetching, permanently skipping a node on a transient export failure); logs use 'stream progress/summary' uniformly and the FetchIncremental path gains per-100 progress + tally summary logs. Structural prep for package split (design §3.1): anonymous Data structs in types.go/blocks.go are named so tests can construct wire types across packages. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(datasource): split feishu connector into core/wiki/drive packages Move shared code (Client, blocks, markdown, types, region, engine, shared helpers) into feishu/core; wiki and drive connectors into their own packages. Export core symbols (capitalize) so wiki/drive can reference them; wiki/drive import core and prefix cross-package references. container.go registration updated to import core/wiki/drive (NewConnector from wiki, NewDriveConnector from drive, Region* from core). Tests are NOT migrated yet (left in feishu/ root, package feishu) and will fail to compile until moved + adapted in a follow-up commit. Source builds clean (go build ./...feishu/... passes). Co-Authored-By: Claude <noreply@anthropic.com> * test(datasource): migrate feishu tests to core/wiki/drive packages Move 14 test files to their subpackages per design §4. Adapt package declarations, imports, and capitalized core symbol references. Add TestMain (SSRF whitelist) to core and drive. Add local helpers (writeJSON, makeConfig, txt, blk constructors, recordingHandler) where cross-package sharing was not possible. Fix engine.go log wording (summary) clobbered by the capitalization script. All tests pass: core 4s, wiki 13s, drive 3s. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(datasource): reduce feishu core export surface; add drive cursor round-trip test Lower-case 87 core symbols that are not referenced cross-package back to unexported (BlocksToMarkdown, CellToString, response types, BlockType constants, etc.), shrinking core's API surface to only what wiki/drive actually use. Restore NodeOps.Fetch capitalization (interface method must match wikiOps/driveOps implementations). Add TestDriveCursorRoundTrip (design §3.2): driveOps.EncodeCursor/DecodeCursorTimes were new code with no direct test. All tests pass: core 3.5s, wiki 13s, drive 3s. Co-Authored-By: Claude <noreply@anthropic.com> * fix(datasource): repair feishu cleanup fallout Restore core.RegionLarkDrive (cleanup lower-cased it; only container.go references it, outside wiki/drive scan). Restore wiki/connector_realapi_test.go local collectHandler type (cleanup wrongly prefixed core.). Apply gofumpt to helpers_test.go. Add open.feishu.cn/open.larksuite.com to test SSRF whitelist: BaseURL-default tests call ParseFeishuConfig which SSRF-validates the resolved hostname; under a proxy open.feishu.cn resolves to 198.18.0.0/15 restricted range. All tests pass: core 3.2s, wiki 13s, drive 2.6s. make build-lite passes. Co-Authored-By: Claude <noreply@anthropic.com> * feat(frontend): feishu drive data source UI and i18n Add Drive folder/token picker strings and data source editor entries for the feishu_drive/lark_drive connectors across en-US/zh-CN/ko-KR/ru-RU. Co-Authored-By: Claude <noreply@anthropic.com> * chore: add trellis journal merge=union rule Append-only developer journals merge cleanly across parallel sessions; task state lives in task.json, not the journal. Co-Authored-By: Claude <noreply@anthropic.com> * refactor(datasource): rename feishu/drive drive_connector.go to connector.go Aligns naming with feishu/wiki subpackage (connector.go). Co-Authored-By: Claude <noreply@anthropic.com> * fix(frontend): allow data:image base64 URIs in sanitized markdown DOMPurify's ALLOWED_URI_REGEXP rejected data: URIs, so <img src="data:image/..."> in uploaded markdown files had their src stripped during sanitizeHTML — the file preview showed broken images while the chunk view (minio:// via hydrateProtectedFileImages) worked. Add data:image/ to the allowed URI scheme so base64 images render in preview and in LLM answers that embed base64 images. Co-Authored-By: Claude <noreply@anthropic.com> * feat(datasource): add FEISHU_DOCX_PARSE_MODE for feishu docx parse path The blocks API path renders image blocks as empty ![图片]() placeholders and fans images into separate knowledge items, breaking image↔document association across retrieval/wiki/agent. FEISHU_DOCX_PARSE_MODE=export forces the async export API (.docx binary -> docreader), so images are parsed inline and bound to the parent document via parent_chunk_id (same as a regular docx upload). Default (unset / "blocks") keeps the existing blocks-first behaviour. Affects both wiki and drive connectors since they share FetchDocxWithBlocks. Co-Authored-By: Claude <noreply@anthropic.com> * feat(datasource): default FEISHU_DOCX_PARSE_MODE to export Switch the feishu docx parse default from blocks to export so embedded images are parsed inline and bound to the parent document via parent_chunk_id (retrieval / wiki / agent all associate images). Operators who need the blocks behaviour (faster, keeps docx attachments, no docreader round-trip) set FEISHU_DOCX_PARSE_MODE=blocks. - shared.go: default unset -> export; update comment to match - .env.example / docker-compose.yml: default export - tests: pin FEISHU_DOCX_PARSE_MODE=blocks on blocks-behaviour tests so they don't break under the new default Co-Authored-By: Claude <noreply@anthropic.com> * docs(datasource): document FEISHU_DOCX_PARSE_MODE and refresh feishu source layout - 飞书云盘接入说明: add section on docx parse mode (export default vs blocks), permission differences, export tradeoffs - 数据源导入开发: add env var + mode comparison - 数据源导入开发文档: rewrite content fetch flow (blocks/export paths), add pros/cons table, fix source-file table to core/wiki/drive layout Co-Authored-By: Claude <noreply@anthropic.com> * fix: 修复合并main后丢失新增的国际化问题 * fix(i18n): 修复误删的国际化内容 * docs: 删除误提交的文件 --------- Co-authored-by: Claude <noreply@anthropic.com>
lyingbug
force-pushed
the
cursor/update-website-docs-bab4
branch
from
August 5, 2026 13:15
6df072c to
9a87dc4
Compare
- Feature contracts: trusted pairs trim overlaps, join adjacent chunks seamlessly, and stay separate across position gaps. - Safety contracts: never drop current content on coordinates alone.
Chunk editing made parser coordinates unreliable, so all merges were downgraded to pure text matching, including unedited documents. Restore position-aware merging for pairs whose coordinates are still trustworthy, and keep the text fallback for edited or stale content so it is never dropped on coordinates.
The heartbeat loop and the detached callback handlers close the client's current connection, but their goroutines can outlive the connection that spawned them: heartbeatCancel only signals, and callback handlers run on a context deliberately detached from the connection. A late-firing goroutine therefore closed whatever connection the reconnect loop had just opened, leaving the client to drop messages until the next read timeout. Pass the owning connection into both callers and close through closeConnIf, which only acts while that connection is still active. This matches the yunzhijia and qqbot drivers, which already scope their heartbeat loops to a single connection. Also fold Stop() into closeConn so it no longer holds c.mu across Close. Stop() was the one caller where the delay actually spread: the IM service tears channels down while holding its channel-map lock, so a TLS close_notify write to a stalled peer stalled every other channel operation. The receive loop's deferred cleanup now closes only through closeConnIf, dropping a redundant second Close of the same connection, and connectAndRun re-checks the closed flag after publishing the connection so a Stop() racing the dial cannot leave a receive loop running until its read deadline expires.
The merge pipeline treats revision 0 as "never edited", so the field fails open to the position path wherever it is not carried over. Make the column explicit for direct row scans and document the remaining JSON gap.
- Overlaps wider than the 400-rune text window must be trimmed once. - Adjacent chunks built from one repeating row must keep every row. - A length-preserving text mismatch must fall back and keep both bodies.
Position-aware merging still routed trimming through AppendWithOverlap, which searches for the longest suffix match inside a window. On periodic text (table rows, log lines) that search mistakes the repetition for the overlap and drops real content, and with a zero overlap there is nothing to search for in the first place. When coordinates are trusted the overlap is already known, so verify it character for character and cut exactly, falling back to the text matcher only when the bodies disagree inside the overlap.
…d knowledge-chat
* chore(types): add internal ContentRewritten to types.SearchResult * test(chunks): pin rewritten results out of the position path Pipeline-replaced content (parent or neighbor expansion) must not trust stale coordinates even when the replacement satisfies the length invariant, and must not extend the merged group's range. * fix(chunks): keep pipeline-rewritten results off the position path Parent and neighbor expansion replace Content after retrieval. Without a marker their stale coordinates could be incorrectly trusted. Ensure ContentRewritten is false in chunkTrusted.
lyingbug
force-pushed
the
cursor/update-website-docs-bab4
branch
from
August 6, 2026 08:08
382b3ef to
bbebc4b
Compare
Import and update WeKnora documentation with VitePress: getting started, architecture, features, API reference, clients, and development guides. Includes landing page with inline SVG illustrations, theme components, doc check scripts, and product screenshots.
Provide demo documents and FAQ import JSON for onboarding tests, plus a minimal HTTP Streamable MCP server for validating WeKnora MCP integration.
lyingbug
force-pushed
the
cursor/update-website-docs-bab4
branch
from
August 6, 2026 10:22
fb7f975 to
08825e0
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
13 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
背景
#2350 的文档是按当时的代码库生成的。本 PR 把它搬到当前 main、逐章对齐代码，并按反馈做了多轮调整。
现共 53 篇。
npm run build通过；npm run check校验内部链接与 Mermaid 语法（141 个链接、61 个图表全部通过）。一、新增章节（原文档缺失）
resource://加载不了」是最常见的问题，答案原先散在 IM 章节、API 总览和配置表里。这一篇讲清四种 URL 形式、各渠道分别取哪种、以及按症状排查的对照表legacy_alias迁移system_settings（DB 盖过 env）config_hash缓存键、埋点与点击归因二、首页改版
原首页讲的是通用 RAG 流程与内部零件计数（14 种解析器、24 个工具），选型的人看不出与同类的差别。改为直说区别在哪：
首页的差异化能力卡片
三、修 Mermaid 报错 + 加校验
两个时序图在浏览器里报
Parse error。根因是消息文本里的;——Mermaid 里分号是语句分隔符，一行被切成两句。vitepress build不校验图表语法也不校验相对链接，坏的只在浏览器里暴露。新增npm run check：check-mermaid.mjs用 Mermaid 自己的parse()逐个校验并报行号，check-links.mjs校验文档间相对链接。它立刻抓到一处我自己留下的死链，建议接进 CI。四、按最新 main 更新内容
Wiki 操作日志下线（
000077）、分块编辑与版本历史（000078）、文档自定义元数据、知识库文件夹树（000079）、resource_urls=public/RESOURCE_URL_MODE、统一导入扩展名闸门、本地 HTML 解析器与 XLSX 表头模式、语义边界重叠、检索合并弃用字符偏移、web_fetch重写、检索引擎按需重建、mcp-server 2.x 与包名变更（工具 28 → 29）、重排分批与 NVIDIA logit 归一化、评估指标修复、IMresource://图片改写、四条新 CI、AWS S3 默认凭证链。五、事实核查修正
两轮全量核查，逐条回到源码验证再改：
invite_only挡不住 OIDC 首次登录（安全相关）；min=6且不调用ValidatePasswordPolicy；system_settings> 启动配置」，界面改过之后改 env 无效；000063已删掉knowledges.tag_id，文档仍列着；关联表也没有chunk_id；minScore 0.3已失效、reflection_enabled后端不消费、WikiConfig不限 wiki 类型；WEKNORA_LANGUAGE优先级写反、bootstrap 变量描述错误、首次访问是登录页、tenantlessonboarding、Lite 应改SYSTEM_AES_KEY、Elasticsearch 无 compose 服务、WEKNORA_SANDBOX_MODE还有local。六、结构与体裁
Lxxx行号与文件长度标注。七、截图占位与站点布局
<Screenshot>组件在图片缺失时渲染带说明的虚线占位框，把同名图片放进website-docs/public/screenshots/即自动生效，不需要改 Markdown；目前 31 处，清单在 README。表格改为外层容器滚动以消除同页宽窄不一；正文列铺满侧栏与目录之间的空间。To show artifacts inline, enable in settings.