Community
Participate
Working Groups
Currently, when a tclsh interpreter is specified, I believe it scans the sub-directories to find the list of packages that are available. However, due to dynamic nature of tclsh, pkg authors can append new paths to $::auto_path variable which in turn leads to new packages to be found during run time. It is also possible to add new paths to $::auto_path variable by setting TCLLIBPATH env variable. As a result, the dltk environment is not able to discover all the tcl packages and provide code assistance for the same. One approach to fix this problem is to invoke the specified tclsh interpreter, try loading an unknown tcl package which will lead it to discover all the packages available for loading and use [package names] command to discover the correct list of packages. See below: [hemang@hlavana-lnx-wny eclipse]$ tclsh % package names activestate::teapot::link platform ActiveTcl Tcl % package require random-[clock seconds] can't find package random-1196660036 % package names logger counter rcs SOAP::Utils routing Plotchart fileutil::magic::mimetype math::rationalfunctions tie::std::dsource tclDESjr math::roman soapinterop::base SOAP::smtp xmlswitch Shcon img::ico patsgui soundex ... (huge list deleted) % # rename source, load and package cmd and define custom procs for same # so that they can be used to determine which files are loaded for each pkg # (I can provide these procs if needed) # Next load each pkg to figure out files corresponding to each pkg % foreach pkg [package names] { package require $pkg ;# (here package cmd invoked is a custom proc) } Note that interpreter specific env variables (such as TCLLIBPATH) will have to be set for the above if it is discover all packages correctly. Hemang.
Hi Hemang, Actually we use paths returned by following script: puts $auto_path As I've understand we need to try load some unknown tcl package, and then return paths?
Loading unknown package will get you one step closer to the solution but it will still not find all the tcl packages available during runtime. Here are two scenarios: 1. The following pkgIndex.tcl file reside under <tcl_root>/lib/ats directory however the actual source files for the pkg reside in some other location: [hemang@hlavana-lnx-wny ats]$ pwd /home/hemang/ats5.0.0/lib/ats [hemang@hlavana-lnx-wny ats]$ cat pkgIndex.tcl if {[package vsatisfies [package provide Tcl] 8.4]} { package ifneeded Async 5.0.0 [string map [list @ [file normalize [file join $dir {../../ats_lib/async}]]] { # ACTIVESTATE TEAPOT-PKG BEGIN REQUIREMENTS package require Atslog package require Cisco package require comm 4.1 package require logger package require parse_dashed_args package require PatsCmd package require Tclx # ACTIVESTATE TEAPOT-PKG END REQUIREMENTS source [file join {@} lib/async.exp] # ACTIVESTATE TEAPOT-PKG BEGIN DECLARE package provide Async 5.0.0 # ACTIVESTATE TEAPOT-PKG END DECLARE }] } [hemang@hlavana-lnx-wny ats]$ In above, the actual source code reside under /home/hemang/ats5.0.0/ats_lib/async/lib directory which tclsh is able to find but dltk doesn't recognize it. 2. In the following case, the pkgIndex.tcl file is designed to recursively look into all its sub-directories and find tcl packages: [hemang@hlavana-lnx-wny ]$ cat pkgIndex.tcl ## pkgIndex.tcl # Manually created, do NOT overwrite (ie. auto-generate) using # tcl's command mk_pkgIndex # Protect from sourcing twice global __ats_pkgIndex_sourced if {![info exists __ats_pkgIndex_sourced($dir)]} { set __ats_pkgIndex_sourced($dir) 1 # Routine used to recursively decend through directories # source package index files into the global context proc processDir {dirname {v ""}} { # Create a cache array variable for saving filenames if {![string equal $v ""]} { upvar 1 $v cache } # Process all the files in the current directory foreach filename [glob -nocomplain [file join $dirname *]] { # Save the filepath in cache so that we do # not recursively process the same directory/file again. if {[info exists cache($filename)]} { continue } set cache($filename) $filename # If a directory is found, recursively process it if {[file isdirectory $filename]} { processDir $filename cache } elseif {[file tail $filename] == "pkgIndex.tcl" && [file exists $filename] && [file readable $filename]} { # Set $dir global variable which is expected to be defined # when pkgIndex.tcl file is sourced. uplevel #0 [list set dir [file dirname $filename]] # Load a package index file uplevel #0 [list source $filename] } } return 1 } processDir $dir rename processDir {} } ;# End of if stmt ## pkgIndex.tcl [hemang@hlavana-lnx-wny ]$ Again, dltk is not able to find packages defined in its sub-directories. In summary: it gets tricky to correctly find out the list of packages that are available for an interpreter. Hence what I am suggesting is that if you explicitly try to load each pkg using "catch {package require <pkg>}", then it is possible to get the full list of packages that are available and the corresponding source files for each package as well. I will try to send you (bit later) a sample script that gives you list of packages that are available and the corresponding src files.
Created attachment 84356 [details] Script to print list of pkgs and corresponding tcl src files I am attaching a script which will find all the packages that are available for the tclsh interpreter and prints an output "pkgname: <files>" for each pkg available to it.
Hi Hemang, I've have some problems with running proposed script. Running tclsh with script fail with general protection error. This happend with ats 5.0a7 destribution (linux version). With original tclsh it works fine. I've attached console output. Next DLTK build will contain both scripts (proposed and original). If proposed script fail, original will be executed.
Created attachment 84411 [details] Console output after running script
Created attachment 86297 [details] Path locator I've modified proposed script to output all paths.
Created attachment 86313 [details] optimized path.tcl file Ok, the latest script looks closer in functionality. As discussed in separate e-mail, the correct solution would be to introduce something like "Built-in" libraries for the interpreter and use the original method to get list of packages and corresponding files. For now, this will suffice. I am also updating the path.tcl script in two respects: 1. Optimizing your add-path routine as follows a. rename add-path to pkg-add-path b. changed roots global variable to pkg_paths array c. optimized performance by removing lsearch operation (use tcl array's inherent properties to do this much faster) d. use [file subcmd] operations to get the directory name 2. Original script used [file normalize] cmd to get cannonical path name. The normalize sub-cmd was introduced in tcl8.4 version. So added a version check for 8.4 before invoking [file normalize] cmd. So this script should work for older tcl versions as well.
Created attachment 86465 [details] More minor speedups I have added some more optimizations: 1. Removed two lsort calls which were not really needed unless those values need to be displayed to the user which we don't need in this case. 2. Some pkgs would rename exit cmd and hence result in taking extra couple of secs to merely terminate this script. So rename exit to exit-org and call exit-org directly which save couple of seconds.
bulk change: downgrading 1.0 to 0.95
