类加载

动态加载dex和so

https://alibaba.github.io/atlas/update/principle.html

PathClassLoader自身负责主Apk的类和c库的查找路口;其parent BootClassloader负责framework sdk的内容的查找。

PathClassLoader本身也是一个class,继承了BaseDexClassLoader(同DexClassLoader),里面查找过程在DexPathList里面实现(如下图) >

image

DexPathList最终通过DexFile去loadClass,DexPathList可以理解为持有者DexFile以及nativeLibrary目录,再查找的时候遍历这些对象,直到找到需要的类或者c库,那么动态部署的方式就是把新修改的内容添加到这些对象的最前面,从而使得查找的过程中新修改的内容能够提前找到从而替换原有的(如下图)

image

BaseDexClassLoader构造方法

private final DexPathList pathList;
/**
 \* Constructs an instance.
 \* Note that all the *.jar and *.apk files from {@code dexPath} might be
 \* first extracted in-memory before the code is loaded. This can be avoided
 \* by passing raw dex files (*.dex) in the {@code dexPath}.
 *
 \* @param dexPath the list of jar/apk files containing classes and
 \* resources, delimited by {@code File.pathSeparator}, which
 \* defaults to {@code ":"} on Android.
 \* @param optimizedDirectory this parameter is deprecated and has no effect
 \* @param librarySearchPath the list of directories containing native
 \* libraries, delimited by {@code File.pathSeparator}; may be
 \* {@code null}
 \* @param parent the parent class loader
 */
 public BaseDexClassLoader(String dexPath, File optimizedDirectory,
     String librarySearchPath, ClassLoader parent) {
   super(parent);
   this.pathList = new DexPathList(this, dexPath, librarySearchPath, null);

   if (reporter != null) {
     reporter.report(this.pathList.getDexPaths());
   }
 }

DexPathList构造方法

/** class definition context */
private final ClassLoader definingContext;
 
 /**
 \* List of dex/resource (class path) elements.
 \* Should be called pathElements, but the Facebook app uses reflection
 \* to modify 'dexElements' (http://b/7726934).
 */
private Element[] dexElements;//dexElements:描述defingContext ClassLoader所加载的dex文件

private final Element[] nativeLibraryPathElements;

//ClassLoader除了加载类之外,加载native动态库的工作也可由它来完成。下面这个变量描述了definingContext ClassLoader可从哪几个目录中搜索目标动态库
private final List<File> nativeLibraryDirectories;
/**
 \* Constructs an instance.
 *
 \* @param definingContext the context in which any as-yet unresolved
 \* classes should be defined
 \* @param dexPath list of dex/resource path elements, separated by
 \* {@code File.pathSeparator}
 \* @param librarySearchPath list of native library directory path elements,
 \* separated by {@code File.pathSeparator}
 \* @param optimizedDirectory directory where optimized {@code .dex} files
 \* should be found and written to, or {@code null} to use the default
 \* system directory for same
 */
 public DexPathList(ClassLoader definingContext, String dexPath,
     String librarySearchPath, File optimizedDirectory) {

// save dexPath for BaseDexClassLoader
 this.dexElements = makeDexElements(splitDexPath(dexPath), optimizedDirectory,
                  suppressedExceptions, definingContext);

DexPathList.makePathElements

private static Element[] makePathElements(List<File> files, File optimizedDirectory,
        List<IOException> suppressedExceptions) {
    return makeDexElements(files, optimizedDirectory, suppressedExceptions, null);
    
/**
 \* Makes an array of dex/resource path elements, one per element of
 \* the given array.
 */
private static Element[] makeDexElements(List<File> files, File optimizedDirectory,
     List<IOException> suppressedExceptions, ClassLoader loader) {
Element[] elements = new Element[files.size()];
int elementsPos = 0;
for (File file : files) {
if (name.endsWith(DEX_SUFFIX)) {
   // Raw dex file (not inside a zip/jar).
   try {
     DexFile dex = loadDexFile(file, optimizedDirectory, loader, elements);
     if (dex != null) {
       elements[elementsPos++] = new Element(dex, null);
     }
}
return elements;

Element构造方法

/**
 \* Element of the dex/resource path. Note: should be called DexElement, but apps reflect on
 \* this.
 */
 /*package*/ static class Element {
   /**
   \* A file denoting a zip file (in case of a resource jar or a dex jar), or a directory
   \* (only when dexFile is null).
   */
   private final File path;
   private final DexFile dexFile;

/**
 \* Element encapsulates a dex file. This may be a plain dex file (in which case dexZipPath
 \* should be null), or a jar (in which case dexZipPath should denote the zip file).
 */
 public Element(DexFile dexFile, File dexZipPath) {
   this.dexFile = dexFile;
   this.path = dexZipPath;
 }
/**
 \* Constructs a {@code DexFile} instance, as appropriate depending on whether
 \* {@code optimizedDirectory} is {@code null}. An application image file may be associated with the {@code loader} if it is not null.
 */
 private static DexFile loadDexFile(File file, File optimizedDirectory, ClassLoader loader,
                  Element[] elements) throws IOException {
   if (optimizedDirectory == null) {
     return new DexFile(file, loader, elements);
   } else {
     String optimizedPath = optimizedPathFor(file, optimizedDirectory);
     return DexFile.loadDex(file.getPath(), optimizedPath, 0, loader, elements);
   }
 }

DexFile.loadDex

    static DexFile loadDex(String sourcePathName, String outputPathName,
        int flags, ClassLoader loader, DexPathList.Element[] elements) throws IOException {
        return new DexFile(sourcePathName, outputPathName, flags, loader, elements);
    }

DexFile构造方法

/*
 \* Private version with class loader argument.
 *
 \* @param file
 \*      the File object referencing the actual DEX file
 \* @param loader
 \*      the class loader object creating the DEX file object
 \* @param elements
 \*      the temporary dex path list elements from DexPathList.makeElements
 */
 DexFile(File file, ClassLoader loader, DexPathList.Element[] elements)
     throws IOException {
   this(file.getPath(), loader, elements);
 }


DexFile(String fileName, ClassLoader loader, DexPathList.Element[] elements) throws IOException {
   mCookie = openDexFile(fileName, null, 0, loader, elements);
   mInternalCookie = mCookie;
   mFileName = fileName;
   //System.out.println("DEX FILE cookie is " + mCookie + " fileName=" + fileName);
 }
/*
 \* Open a DEX file. The value returned is a magic VM cookie. On
 \* failure, an IOException is thrown.
 */
 private static Object openDexFile(String sourceName, String outputName, int flags,
     ClassLoader loader, DexPathList.Element[] elements) throws IOException {
   // Use absolute paths to enable the use of relative paths when testing on host.
   return openDexFileNative(new File(sourceName).getAbsolutePath(),
               (outputName == null)? null: new File(outputName).getAbsolutePath(),
               flags,
               loader,
               elements);
 }

/art/runtime/native/dalvik_system_DexFile.cc

267static jobject DexFile_openDexFileNative(JNIEnv* env,
268                                         jclass,
269                                         jstring javaSourceName,
270                                         jstring javaOutputName ATTRIBUTE_UNUSED,
271                                         jint flags ATTRIBUTE_UNUSED,
272                                         jobject class_loader,
273                                         jobjectArray dex_elements) {

ScopedUtfChars sourceName(env, javaSourceName);

279  Runtime* const runtime = Runtime::Current();
280  ClassLinker* linker = runtime->GetClassLinker();
281  std::vector<std::unique_ptr<const DexFile>> dex_files;
282  std::vector<std::string> error_msgs;
283  const OatFile* oat_file = nullptr;
  //main
285  dex_files = runtime->GetOatFileManager().OpenDexFilesFromOat(sourceName.c_str(),
286                                                               class_loader,
287                                                               dex_elements,
288                                                               /*out*/ &oat_file,
289                                                               /*out*/ &error_msgs);

291  if (!dex_files.empty()) {
292    jlongArray array = ConvertDexFilesToJavaArray(env, oat_file, dex_files);
293    if (array == nullptr) {
294      ScopedObjectAccess soa(env);
295      for (auto& dex_file : dex_files) {
296        if (linker->IsDexFileRegistered(soa.Self(), *dex_file)) {
297          dex_file.release();
298        }
299      }
300    }
301    return array;

/art/runtime/oat_file_manager.h or cc

OatFileManager::OpenDexFilesFromOat

84  // Finds or creates the oat file holding dex_location. Then loads and returns
85  // all corresponding dex files (there may be more than one dex file loaded
86  // in the case of multidex).
87  // This may return the original, unquickened dex files if the oat file could
88  // not be generated.
89  //
90  // Returns an empty vector if the dex files could not be loaded. In this
91  // case, there will be at least one error message returned describing why no
92  // dex files could not be loaded. The 'error_msgs' argument must not be
93  // null, regardless of whether there is an error or not.
94  //
95  // This method should not be called with the mutator_lock_ held, because it
96  // could end up starving GC if we need to generate or relocate any oat files.
394std::vector<std::unique_ptr<const DexFile>> OatFileManager::OpenDexFilesFromOat(
395    const char* dex_location,
396    jobject class_loader,
397    jobjectArray dex_elements,
398    const OatFile** out_oat_file,
399    std::vector<std::string>* error_msgs) {

404  // Verify we aren't holding the mutator lock, which could starve GC if we
405  // have to generate or relocate an oat file.
406  Thread* const self = Thread::Current();
407  Locks::mutator_lock_->AssertNotHeld(self);
408  Runtime* const runtime = Runtime::Current();

410  std::unique_ptr<ClassLoaderContext> context;
411  // If the class_loader is null there's not much we can do. This happens if a dex files is loaded
412  // directly with DexFile APIs instead of using class loaders.
413  if (class_loader == nullptr) {
416    context = nullptr;
417  } else {
418    context = ClassLoaderContext::CreateContextForClassLoader(class_loader, dex_elements);
419  }
420
421  OatFileAssistant oat_file_assistant(dex_location,
422                                      kRuntimeISA,
423                                      !runtime->IsAotCompiler(),
424                                      only_use_system_oat_files_);

437  if (!oat_file_assistant.IsUpToDate()) {
//main
oat_file_assistant.MakeUpToDate(/*profile_changed*/ false,
452                                            actual_context,
453                                            /*out*/ &error_msg)

/art/libdexfile/dex/dex_file.h

class DexFile

63// Dex file is the API that exposes native dex files (ordinary dex files) and CompactDex.
64// Originally, the dex file format used by ART was mostly the same as APKs. The only change was
65// quickened opcodes and layout optimizations.
66// Since ART needs to support both native dex files and CompactDex files, the DexFile interface
67// provides an abstraction to facilitate this.
68class DexFile {
83  // Raw header_item.
84  struct Header {
85    uint8_t magic_[8] = {};
86    uint32_t checksum_ = 0;  // See also location_checksum_
87    uint8_t signature_[kSha1DigestSize] = {};
88    uint32_t file_size_ = 0;  // size of entire file
89    uint32_t header_size_ = 0;  // offset to start of next section
90    uint32_t endian_tag_ = 0;
91    uint32_t link_size_ = 0;  // unused
92    uint32_t link_off_ = 0;  // unused
93    uint32_t map_off_ = 0;  // unused
94    uint32_t string_ids_size_ = 0;  // number of StringIds
95    uint32_t string_ids_off_ = 0;  // file offset of StringIds array
96    uint32_t type_ids_size_ = 0;  // number of TypeIds, we don't support more than 65535
97    uint32_t type_ids_off_ = 0;  // file offset of TypeIds array
98    uint32_t proto_ids_size_ = 0;  // number of ProtoIds, we don't support more than 65535
99    uint32_t proto_ids_off_ = 0;  // file offset of ProtoIds array
100    uint32_t field_ids_size_ = 0;  // number of FieldIds
101    uint32_t field_ids_off_ = 0;  // file offset of FieldIds array
102    uint32_t method_ids_size_ = 0;  // number of MethodIds
103    uint32_t method_ids_off_ = 0;  // file offset of MethodIds array
104    uint32_t class_defs_size_ = 0;  // number of ClassDefs
105    uint32_t class_defs_off_ = 0;  // file offset of ClassDef array
106    uint32_t data_size_ = 0;  // size of data section
107    uint32_t data_off_ = 0;  // file offset of data section
108
109    // Decode the dex magic version
110    uint32_t GetVersion() const;
111  };

/art/runtime/oat_file_assistant.cc

OatFileAssistant::MakeUpToDate

251OatFileAssistant::MakeUpToDate(bool profile_changed,
252                               ClassLoaderContext* class_loader_context,
253                               std::string* error_msg) {

262  OatFileInfo& info = GetBestInfo();

273  switch (info.GetDexOptNeeded(
274        target, profile_changed, /*downgrade*/ false, class_loader_context)) {
275    case kNoDexOptNeeded:
276      return kUpdateSucceeded;
277
278    // TODO: For now, don't bother with all the different ways we can call
279    // dex2oat to generate the oat file. Always generate the oat file as if it
280    // were kDex2OatFromScratch.
281    case kDex2OatFromScratch:
282    case kDex2OatForBootImage:
283    case kDex2OatForRelocation:
284    case kDex2OatForFilter:
285      return GenerateOatFileNoChecks(info, target, class_loader_context, error_msg);
286  }

OatFileAssistant::GenerateOatFileNoChecks

698OatFileAssistant::ResultOfAttemptToUpdate OatFileAssistant::GenerateOatFileNoChecks(
699      OatFileAssistant::OatFileInfo& info,
700      CompilerFilter::Filter filter,
701      const ClassLoaderContext* class_loader_context,
702      std::string* error_msg) {

717  const std::string& oat_file_name = *info.Filename();
718  const std::string& vdex_file_name = GetVdexFilename(oat_file_name);

43  Dex2oatFileWrapper vdex_file_wrapper(OS::CreateEmptyFile(vdex_file_name.c_str()));
744  File* vdex_file = vdex_file_wrapper.GetFile();

759  Dex2oatFileWrapper oat_file_wrapper(OS::CreateEmptyFile(oat_file_name.c_str()));
760  File* oat_file = oat_file_wrapper.GetFile();

773  std::vector<std::string> args;
774  args.push_back("--dex-file=" + dex_location_);
775  args.push_back("--output-vdex-fd=" + std::to_string(vdex_file->Fd()));
776  args.push_back("--oat-fd=" + std::to_string(oat_file->Fd()));
777  args.push_back("--oat-location=" + oat_file_name);
778  args.push_back("--compiler-filter=" + CompilerFilter::NameOfFilter(filter));
779  const std::string dex2oat_context = class_loader_context == nullptr
780        ? OatFile::kSpecialSharedLibrary
781        : class_loader_context->EncodeContextForDex2oat(/*base_dir*/ "");
782  args.push_back("--class-loader-context=" + dex2oat_context);

784  if (!Dex2Oat(args, error_msg)) {//main
785    return kUpdateFailed;
786  }
787
788  if (vdex_file->FlushCloseOrErase() != 0) {
789    *error_msg = "Unable to close vdex file " + vdex_file_name;
790    return kUpdateFailed;
791  }
792
793  if (oat_file->FlushCloseOrErase() != 0) {
794    *error_msg = "Unable to close oat file " + oat_file_name;
795    return kUpdateFailed;
796  }

804  return kUpdateSucceeded;

OatFileAssistant::Dex2Oat

807bool OatFileAssistant::Dex2Oat(const std::vector<std::string>& args,
808                               std::string* error_msg) {

816  std::vector<std::string> argv;
817  argv.push_back(runtime->GetCompilerExecutable());

849  argv.insert(argv.end(), args.begin(), args.end());
850
851  std::string command_line(android::base::Join(argv, ' '));
852  return Exec(argv, error_msg);

/art/runtime/runtime.h

723std::string Runtime::GetCompilerExecutable() const {
724  if (!compiler_executable_.empty()) {
725    return compiler_executable_;
726  }
727  std::string compiler_executable(GetAndroidRoot()); //getenv("ANDROID_ROOT")
728  compiler_executable += (kIsDebugBuild ? "/bin/dex2oatd" : "/bin/dex2oat");
//位于设备system/bin/dex2oat
729  return compiler_executable;
730}

/art/dex2oat/dex2oat.cc

3176int main(int argc, char** argv) {
3177  int result = static_cast<int>(art::Dex2oat(argc, argv));
3184  return result;
3185}

具体参考类编译原理

classLoader.loadClass

方法调用时机

对于APPComponentFactory子类,LoadedApk进行调用

对于application和四大组件由instrumentation调用APPComponentFactory进行调用

对于view,由LayoutInflator在createView方法中进行调用

对于其他自定义类,由jvm进行调用

Class objects for array classes are not created by class loaders, but are created automatically as required by the Java runtime. The class loader for an array class, as returned by Class#getClassLoader() is the same as the class loader for its element type; if the element type is a primitive type, then the array class has no class loader.

protected Class<?> loadClass(String name, boolean resolve)
   throws ClassNotFoundException
 {
// First, check if the class has already been loaded
 Class<?> c = findLoadedClass(name);
 if (c == null) {
   try {
     if (parent != null) {
       c = parent.loadClass(name, false);
     } else {
       c = findBootstrapClassOrNull(name);
     }
   } catch (ClassNotFoundException e) {
     // ClassNotFoundException thrown if class not found
     // from the non-null parent class loader
   }
   if (c == null) {
     // If still not found, then invoke findClass in order
     // to find the class.
     c = findClass(name);//main
   }
 }
 return c;
 @Override
 protected Class<?> findClass(String name) throws ClassNotFoundException {
   return Class.classForName(name, false, null);
 }
/** Called after security checks have been made. */
 @FastNative
 static native Class<?> classForName(String className, boolean shouldInitialize,
     ClassLoader classLoader) throws ClassNotFoundException;

BootClassLoader.findClass

class BootClassLoader extends ClassLoader {
    //BootClassLoader采用了单例构造的方式。所以一个Java进程只存在一个BootClassLoader对象
    private static BootClassLoader instance;
    //Loader没有可委托的其他加载器了。
    public BootClassLoader() { super(null); }
    //加载目标类,下文将分析其代码。
    protected Class<?> findClass(String name) ... {
        return Class.classForName(name, false, null);
    }
}

art/runtime/native/java_lang_Class.cc

// "name" is in "binary name" format, e.g. "dalvik.system.Debug$1".
static jclass Class_classForName(JNIEnv* env, jclass, jstring javaName,
        jboolean initialize,jobject javaLoader) {
    //注意传入的参数,javaName为JLS规范里定义的类名(如java.lang.String),
    //initialize为false,javaLoader为false
    ScopedFastNativeObjectAccess soa(env);
    ScopedUtfChars name(env, javaName);
    ......
    //转成JVM规范使用的类名,如Ljava/lang/String;
    std::string descriptor(DotToDescriptor(name.c_str()));
    StackHandleScope<2> hs(soa.Self());
   
    Handle<mirror::ClassLoader> class_loader(hs.NewHandle(
        soa.Decode<mirror::ClassLoader*>(javaLoader)));
    ClassLinker* class_linker = Runtime::Current()->GetClassLinker();
 
    Handle<mirror::Class> c(
        hs.NewHandle(class_linker->FindClass(soa.Self(),//main
            descriptor.c_str(), class_loader)));
    ......
    if (initialize) {      //initialize为false的话,将只加载和链接目标类,不初始化它
        class_linker->EnsureInitialized(soa.Self(), c, true, true);
    }
    return soa.AddLocalReference<jclass>(c.Get());
}

BaseDexClassLoader.findClass

@Override
 protected Class<?> findClass(String name) throws ClassNotFoundException {
   List<Throwable> suppressedExceptions = new ArrayList<Throwable>();
   Class c = pathList.findClass(name, suppressedExceptions);
  return c;

DexPathList.findClass

/**
 \* Finds the named class in one of the dex files pointed at by
 \* this instance. This will find the one in the earliest listed
 \* path element. If the class is found but has not yet been
 \* defined, then this method will define it in the defining
 \* context that this instance was constructed with.
 *
 \* @param name of class to find
 \* @param suppressed exceptions encountered whilst finding the class
 \* @return the named class or {@code null} if the class is not
 \* found in any of the dex files
 */
 public Class<?> findClass(String name, List<Throwable> suppressed) {
   for (Element element : dexElements) {
     Class<?> clazz = element.findClass(name, definingContext, suppressed);
     if (clazz != null) {
       return clazz;
     }
   }
public Class<?> findClass(String name, ClassLoader definingContext,
     List<Throwable> suppressed) {
   return dexFile != null ? dexFile.loadClassBinaryName(name, definingContext, suppressed)
       : null;
 }

/libcore/dalvik/src/main/java/dalvik/system/DexFile.java

DexFile.defineClass

/**
 \* See {@link #loadClass(String, ClassLoader)}.
  \* This takes a "binary" class name to better match ClassLoader semantics.
 */
 public Class loadClassBinaryName(String name, ClassLoader loader, List<Throwable> suppressed) {
   return defineClass(name, loader, mCookie, this, suppressed);
 }
private static Class defineClass(String name, ClassLoader loader, Object cookie,
                 DexFile dexFile, List<Throwable> suppressed) {
   Class result = null;
   try {
     result = defineClassNative(name, loader, cookie, dexFile);

art/runtime/native/dalvik_system_DexFile.cc

static jclass DexFile_defineClassNative(JNIEnv* env,
                                        jclass,
                                        jstring javaName,
                                        jobject javaLoader,
                                        jobject cookie,
                                        jobject dexFile) {

      ObjPtr<mirror::Class> result = class_linker->DefineClass(soa.Self(),
                                                               descriptor.c_str(),
                                                               hash,
                                                               class_loader,
                                                               *dex_file,
                                                               *dex_class_def);
      // Add the used dex file. This only required for the DexFile.loadClass API since normal
      // class loaders already keep their dex files live.
      class_linker->InsertDexFileInToClassLoader(soa.Decode<mirror::Object>(dexFile),
                                                 class_loader.Get());


PathClassLoader和DexClassLoader区别

两种ClassLoader区别在于DexClassLoader可以加载外置卡中的apk,而PathClassLoader用于加载已安装的apk,构造参数里只有优化路径的参数差异,PathClassLoader传入的是null,而DexClassLoader是可以传入的,这个差异最终是在oat_file_assistant.cc这个文件里的MakeUpToDate方法中进行判断处理,如果noDexOptNeeded(已安装apk)则直接返回,否则(未安装的apk)会进行Dex2Oat过程,这个过程需要这个优化路径,否则无法加载

总结

https://www.jianshu.com/p/2216554d3291

BaseDexClassLoader 的构造函数中创建一个DexPathList实例,DexPathList的构造函数会创建一个dexElements 数组

BaseDexClassLoader 在findclass方法中调用了pathList.findClass,这个方法中会遍历dexpathlist中的dexElements数组,如果DexFile不为空那么调用DexFile类的loadClassBinaryName方法返回Class实例。

Tinker进行热修复的流程为:

新dex与旧dex通过dex差分算法生成差异包 patch.dex

将patch dex下发到客户端,客户端将patch dex与旧dex合成为新的全量dex

将合成后的全量dex 插入到dex elements前面(此部分和QQ空间机制类似),完成修复

可见,Tinker和QQ空间方案最大的不同是,Tinker 下发新旧DEX的差异包,然后将差异包和旧包合成新dex之后进行dex的全量替换,这样也就避免了QQ空间中的插桩操作。

https://blog.csdn.net/u010386612/article/details/51077291

http://gityuan.com/2017/03/19/android-classloader/